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ABSTRACT : 


A  methodology  is  developed  for  auditing  cost  effectiveness 
analyses  of  major  technological  changes.   The  methodology 
is  applied  to  the  Work  In  Process  Inventory  Control  System 
(WIPICS)  recently  implemented  at  NARF,  North  Island.   The 
approach  involves  using  data  on  NARF  operations  to  estimate 
cost  functions  for  each  major  program  of  the  NARF  both 
before  and  after  the  change.   Cost  comparisons  using  these 
models  do  not  show  a  clear  cost  savings  for  the  WIPICS 
system. 
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and  in  terms  of  a  practical  use  on  a  particular  technological  change. 
Since  major  portions  of  the  report  concern  data  and  specific  circum- 
stances associated  with  the  practical  example,  it  is  useful  to  detail 
the  particular  technological  change  involved  at  this  point. 
B.   AN  EXAMPLE  OF  TECHNOLOGICAL  CHANGE 

The  Naval  Air  Research  Facility  at  NAS  North  Island  (NARFNI)  is  one 
of  seven  naval  air  rework  facilities  throughout  the  continental  United 
States  responsible  for  major  maintenance,  conversion  and  repair  of  United 
States  Navy  and  Marine  Corps  aircraft  and  related  components.   To 
accomplish  this  mission  NARFNI  employes  approximately  6800  civilian 
workers  and  spends  $150  million  on  annual  operating  expenses. 

Maintenance  of  equipment  is  accomplished  under  one  of  three  primary 
programs,  with  two  of  the  programs  receiving  inputs  either  directly  from 
the  customer  or  from  the  third  program  (see  Fig.  1). 
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FIGURE  1.   NARFNI  Repair  Programs 

Under  these  programs  work  is  further  assigned  to  one  of  the  six 
divisions  of  the  Production  Department  identified  in  Figure  2. 


I.   INTRODUCTION  AND  BACKGROUND 
A.   INTRODUCTION 

It  is  commonly  accepted  that  technological  advancements  are 
arriving  at  a  faster  and  faster  pace  with  the  state  of  the  art  generally 
far  ahead  of  the  functional  application.   One  reason  for  this  gap  is 
the  costly  enterprise  of  transfering  theory  into  practice.   When  the 
initial  transfer  is  made  a  natural  question  to  ask  prior  to  extensive 
implementation  is  "Will  it  be  cost-effective?"   Current  methods  of 
cost-effectiveness  analysis  are  tedious  and  usually  employ  a  detailed 
analysis  of  the  process  which  incorporated  the  change.   Such  analyses 
are  inherently  imprecise  due  to  the  possibilities  of  overlooked  depen- 
dencies, double  counting  of  costs,  and  external  changes  which  may  affect 
the  effectiveness  of  the  process  and  yet  not  be  accounted  for  in  the 
analysis.   Thus  when  a  cost-effectiveness  analysis  is  completed  a 
reviewing  agency  might  well  ask,  "Are  the  conclusions  of  the  analysis 
correct?"   As  mentioned,  the  methods  employed  in  the  analysis  make 
this  second  question  a  difficult  one  to  answer. 

Nevertheless,  it  is  important  to  answer  the  above  question.   Many 
technological  changes  are  implemented  on  a  prototype  basis.   Therefore, 
it  is  increasingly  important  that  a  careful  audit  of  the  cost-effective- 
ness analysis  be  made.   That  is,  the  costs,  cost  savings,  and  increased 
effectiveness  attributable  to  the  technological  change  must  be  carefully 
documented  after  the  installation  of  the  prototype  and  before  similar 
changes  are  implemented  at  other  installations. 

This  report  provides  several  alternative  methods  of  auditing  cost- 
effectiveness  analyses.   It  compares  the  methods  on  a  theoretical  basis 
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FIGURE  2.   Production  Department  Divisions 


The  assignment  of  work  to  various  programs  and  production  divisions 
can  be  thought  of  as  a  "job  shop"  operation  in  which  each  shop  performs 
a  specific  task.   Thus  an  item  entering  rework  is  subject  to  being 
dismantled,  its  subassemblies  distributed  for  rework,  and  then  re- 
assembled prior  to  returning  to  the  customer.   This  type  of  operation 
places  emphasis  on  the  scheduling  of  rework  in  that  ideally  the  shops 
are  completing  work  on  related  parts  in  the  same  sequence  to  reduce  the 
total  repair  time  of  the  major  end  items.   Scheduling  is  also  important 
to  the  shop  managers  who  must  order  material  and  plan  personnel  assign- 
ments based  on  the  projected  arrival  of  work  from  other  shops.   This 
entire  scheduling  problem  is  intimately  related  to  the  technological 
change  to  be  described  in  the  following  section. 

NARFNI  is  primarily  funded  by  the  Navy  Industrial  Fund  (NIF) 
which  provides  a  working  capital  fund  to  finance  repair  operations. 
The  finished  product  is  returned  with  a  bill  to  the  customer  for  the 
work  done.   This  "debt"  is  then  paid  by  transferring  funds  from  the 
customer's  appropriated  maintenance  funds  to  the  NARF's  working  capital 
fund.   Under  this  concept  NARFNI  is  required  to  control  its  finances 
to  incur  zero  profit  at  the  end  of  each  fiscal  year.   This  control  is 


exercised  at  Quarterly  Planning  Conferences  in  which  representatives  from 
the  customers  and  NARFNI  meet  to  plan  the  following  quarter's  work  load 
input  and  prices  for  the  work  to  be  accomplished.   NARFNI  is  reimbursed 
under  basically  two  types  of  contracts,  cost  reimbursable  and  fixed  price, 
the  majority  of  the  work  being  performed  under  fixed  price  contracts 
established  at  the  Quarterly  Planning  Conference.   This  type  of  contract 
gives  NARFNI  incentive  to  minimize  costs  subject  to  the  required  work  load, 
a  fact  which  is  instrumental  to  the  auditing  method  investigated. 

In  January  1972,  NARFNI  installed  an  industrial  information  system, 
the  Work  in  Process  Inventory  Control  System  (WIPICS) ,  to  assist  in  the 
location  and  scheduling  of  inventory  in  the  rework  process.   WIPICS  was 
requested  and  initiated  by  the  Management  Systems  Development  Office  (MSDO) 
in  an  effort  to  assist  the  NARF's  in  performing  their  mission  at  a  minimum 
cost  to  the  government.   If  the  prototype  at  NARFNI  is  determined  to  be 
cost-effective  WIPICS  will  then  be  installed  at  the  remaining  six  NARF's. 
Its  major  milestone  are  listed  in  Figure  3. 


1.  Early  1969  -  ROHR  Corporation  studied  NARFNI 

2.  Late  1969  -  MSDO  requested  contract 

3.  Early  1970  -  NAVAIRSYSCOM  evaluation  and  approval 

4.  July  1970  -  SECNAV  authority  for  prototype 

5.  Late  1970  -  Contract  negotiated 

6.  4  January  1971  -  Contract  date,  D-Day 

7.  D  +  9.5  months  -  WIPICS  developed 

8.  D  +  11  months  -  WIPICS  test 

9.  D  +  11.5  months  -  WIPICS  prototype  started 

FIGURE  3.  WIPICS  Milestone 
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WIPICS  consists  of  a  central  computer  which  is  linked  to  the  job 
shops  by  164  "touch  tone"  telephones,  20  alphanumeric  terminals,  and 
four  teletypewriters.   In  addition,  the  system  is  equipped  with  an  80 
word  audio-response  unit  to  allow  two-way  communication  through  the 
telephones.   When  an  item  first  arrives  at  NARFNI  it  is  broken  down 
into  identifiable  subassemblies.   For  each  of  these  subassemblies  a 
computer  record  is  created  and  entered  into  WIPICS.   This  record  con- 
tains information  such  as  identification,  required  work,  required 
material,  location,  and  status.   Each  job  shop  inputs  arrival  and  de- 
parture  information  through  the  "touch-tone"  telephones.   Also,  an 
unexpected  delay  in  work  due  to  lack  of  material  or  some  other  short- 
coming is  inputed  as  it  occurs.   At  the  completion  of  the  work  day  a 
print-out  of  this  information  is  produced  to  aid  shop  managers  in 
projecting  the  rate  and  mix  of  work  that  will  be  entering  their  respective 
shops  in  the  near  future. 

The  inauguration  of  the  WIPICS  prototype  was  accomplished  by 
creating  computer  records  for  all  jobs  presently  in  the  shop  to  preclude 
a  lengthy  start  up  before  evaluation  could  begin.   Simultaneously,  the 
operational  costs  accountable  to  WIPICS  were  transferred  as  overhead  to 
be  applied  to  the  price  of  jobs  in  shop.   Thus,  during  January,  1972, 
NARFNI  initiated  a  technological  change  which  was  to  be  subjected  to 
cost-effectiveness  analysis  and,  at  the  same  time,  provided  a  suitable 
situation  to  use  in  investigating  a  method  for  auditing  cost-effective- 
ness analyses. 
C.    PREVIOUS  WORK 

This  report  documents  our  research  in  auditing  cost-effectiveness 
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analyses  and  summarizes  several  previous  reports,  presentations  and 
theses  generated  by  the  research  project.   In  particular  six  master's 
theses  in  Operation  Analysis  references  [1,  2,  3,  4,  5,  6]  provide 
much  of  the  detail  and  data  that  underlie  the  report.   A  brief  synopsis 
of  each  follows. 

Spooner  [1]  details  the  general  methodology  to 
be  followed  in  using  continuous  production  functions 
of  the  Cobb-Douglas  variety  to  model  a  multiproduct 
facility  before  and  after  a  technological  change. 
He  also  provides  the  initial  before  WIPICS  data 
on  engine  and  air  frame  jobs  and  a  computer  program 
to  convert  this  information  to  a  flow  of  work  over 
time  at  NARFNI. 

Myers  [2]  developes  linear  economic  models  of  the 
air  frame  and  engine  programs  at  NARFNI.   He  also 
provides  an  estimation  procedure  for  determining 
the  transformation  matrix  from  data  on  jobs. 
Bradley  [3]  uses  data  provided  by  Spooner  to 
estimate  production  functions  for  airframes, 
engines,  and  F/J  components.   He  also  does  some 
preliminary  analysis  of  the  timing  of  fluctuation 
in  workload  at  the  NARF. 

Trafton  [4]  extends  Bradley's  work  by  estimating 
cost  functions  of  the  same  form  as  the  latter 
production  functions.   He  also  develops  and  uses 
an  estimation  procedure  for  the  cost  function 
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derived  from  the  Constant  Elasticity  of  Substitution 
Production  Function. 

Tye  [5]  continues  this  line  of  reasoning  by 
estimating  similar  functions  for  sets  of  data 
collected  after  WIPICS  has  been  implemented.   He 
tests  their  significance,  constructs  prediction 
intervals  and  derives  the  first  preliminary  con- 
clusions concerning  the  cost-effectiveness  of  the 
system. 

Finally  McGarrahan  [6]  develops  models  similar  to 
Myers'  for  the  after  WIPICS  data  and  compares 
the  results  from  these  models  to  those  derived 
by  Tye.   He  also  reports  a  third  type  of  model  and 
compares  it  to  the  other  two  types. 
The  above  theses  contain  data,  computer  programs,  and  specific 
results  which  support  the  information  provided  in  this  report. 

The  report  is  divided  into  several  sections.   The  section  below 
presents  the  methodology  of  the  study  and  the  alternative  mathematical 
models  that  are  employed.   Section  three  describes  the  data  that  were 
used  and  various  transformations  and  adjustments  to  that  data.   In 
section  four  the  various  estimation  procedures  that  are  employed  are 
presented.   The  results  of  those  procedures  are  reported  in  section  five 
along  with  tests  of  hypotheses,  and  prediction  intervals.   Section  six 
of  the  report  contains  comparisons  of  the  various  models,  the  conclusions 
of  the  study,  and  topics  for  further  research. 
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II  METHODOLOGY  AND  SCOPE  OF  THE  PROJECT 
A.    INTRODUCTION 

Usually  the  major  source  of  effectiveness  for  new  equipment  or  pro- 
cedures is  in  terms  of  factor  saving.   That  is,  an  analysis  will  frequently 
document  expected  effectiveness  in  term  of  man-days  saved  or  decreases  in 

wasted  raw  materials  or  replacement  of  several  more  expensive  pieces  of 
equipment. 

Other  contributions  to  effectiveness  are  more  difficult  to  estimate 
and  evaluate.   However,  some  attempt  is  usually  made  to  include  effects 
like  increased  speed  of  production,  higher  quality  of  output,  and  more 
control  over  the  production  process. 

Frequently,  an  attempt  is  made  to  assign  dollar  values  to  each  of 
these  measures  of  effectiveness  and  a  cost-effective  change  is  defined 
as  one  with  a  greater  value  of  effectiveness  than  its  costs. 

Frequently  unstated  in  the  analysis  is  the  assumption  that  these 
effects  are  expected  only  if  nothing  else  changes. 

After  the  equipment  has  been  installed  or  the  procedure  implemented, 
one  can  gather  data  on  the  changes  that  have  taken  place.   For  example, 
we  know  what  has  happened  to  factor  usage,  we  can  measure  the  new  speed 
of  the  production  process,  and  perhaps  we  can  determine  quality  changes 
in  output.   Here  the  important  questions  are  not  what  changes  have  taken 
place,  but  why  the  changes  have  occurred.   In  particular  are  the  changes 
due   to  our  new  equipment  or  procedure  ? 

Notice  that  after  the  fact,  many  other  things  may  have  changed  as 
well.  In  particular,  the  outputs  of  the  organization  may  have  changed 
in  both  quantity  and  type,  prices  of  factors,  many  have  changed  or  new 
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constraints  may  have  been  placed  on  the  organization  or  old  ones  relaxed. 
Finally,  the  manager  of  the  enterprise  may  not  have  acted  exactly  as  the 
cost-effectiveness  analysis  expected.   For  example,  suppose  a  new  piece 
of  equipment  replaces  old  equipment  and  several  men,  instead  of  laying 
off  the  men  the  manager  re-assigns   them  as  trouble  shooters.   As  a 
result,  output  increases  dramatically  with  no  increase  in  costs.   Here 
costs  did  not  decline  as  expected,  instead  the  manager  chose  to  increase 
output. 

Current  methods  of  auditing  cost-effectiveness  analyses  involve 
looking  at  factors  by  categories  and  determining  if  their  usage  has 
changed.   A  portion  of  the  change  in  factor  usage  is  directly  related 
to  the  new  equipment  by  verbal  argument  and  a  dollar  value  is  assigned 
to  that  portion  of  the  change  in  factor  usage.   Thus  the  relation  between 
the  factor  savings  and  the  new  equipment  frequently  is  a  verbal  argument 
constructed  by  an  outside  observer.   These  allowed  effects  are  then 
compared  to  costs.   Other  effects  like  quality  changes  are  handled 

separately. 

Our  proposal  for  auditing  CE  analyses  recognizes  that  an  outside 

observer  cannot  effectively  trace  second  order  effects  though  a  massive 

enterprise.   Thus  the  proposal  is  to  look  at  aggregated  summary  measures 

of  the  organization's  behavior  in  each  of  several  areas  before  and  after 

the  technological  change. 

In  particular  our  approach  to  determining  the  effectiveness  of 

WIPICS  is  to  develop   a  model  of  the  production  behavior  of  the  NARF  both 

before  and  after  the  implementation  of  the  system.   These  production 

models  are  then  compared  for  several  sets  of  circumstances  and  conclusions 
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are  drawn  from  the  comparisons.   Three  different  production  models  have 
been  used  for  these  purposes  and  each  of  them  is  described  below. 
B.    THE  CONTINUOUS  PRODUCTION  FUNCTION  OF  A  MAJOR  PROGRAM 

A  production  function  is  frequently  used  to  describe  the  relations 
between  factors  and  the  homogeneous  output  that  is  produced  by  their  use. 
Ordinarily,  the  function  describes  this  relation  for  a  specified  period 
of  time.   In  addition  the  function  typically  possesses  the  following 
characteristics : 

1.  W  =  f(Z^,Z2,Z3)   where  W  is  output,   Zi,Z2»Z3  are  quantities 
of  factors  used,  and   f   is  a  single  valued,  continuous  function 
whose  first  and  second  derivatives  exist,   f   represents  the 
maximum  output  that  can  be  produced  with  a  given  vector  of 

factors. 

3f 

2.  >  0,   the  marginal  product  of  each  of  the  factors  is  possi- 

tive. 


3.   The  matrix   „„"„^ —   is  negative  semidef inite.   This  requires 
the  rate  at  which  one  factor  may  be  substituted  for  another 
while  producing  a  given  quantity  of  output  diminishes  as  the 
second  factor  is  substituted  for  the  first. 
To  elaborate  on  this  last  property,  it  may  be  possible  to  produce 
a  PAR  Repair  on  an   A3B  with  input  combinations  one  and  two  below,  but 
one  would  hardly  expect  that  the  possibilities  of  substituting  labor  for 
material  at  the  same  rate  would  extend  to  situation  three. 
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Situations 


Variables 

1 

2 

3 

PAR  Repairs 
A3B 

1 

1 

1 

Direct  Man 
Hours 

12,600 

12,000 

10,200 

Material 
Costs 

11,400 

15,000 

25,800 

FIGURE  4.   Production  Tradeoffs 


By  estimating  a  production  function  for  an  entire  program  we  are 
not  involved  with  modeling  the  detailed  relations  among  shops  or  the 
exact  relation  between  WIPICS  and  the  cost  producing  variables  (factors) 
or  the  effectiveness  generating  variables  (outputs).   This  eliminates 
the  need  to  conduct  expensive  and  time  consuming  measurements  of  many 
variables  and  also  prevents  the  possible  double  counting  problems  that 
are  difficult  to  avoid  in  a  cost-effectiveness  analysis. 

On  the  other  hand,  the  production  function  is  sufficiently  detailed 
to  permit  the  analysis  of  the  NARF's  opportunities  for  tradeoffs  among 
factors.   For  example,  the  trade-off  between  man-hours  and  repairable 
airframes  given  some  level  of  output  and  material  usage  is  measured  by  the 
production  function.   Thus,  the  opportunity  to  take  cost  savings  due  to 
WIPICS  in  the  form  of  reduced  man  hours  or  reduced  inventory  of  repairable 
airframes  is  measured. 

The  basic   information  embodied  in  the  production  function  may  be 
combined  with  some  additional  assumptions  concerning  the  organization 
to  determine  a  cost  function  for  a  program. 
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For  example,  it  may  be  assumed  that  the  NARF  attempts  to  minimize 
the  cost  of  producing  a  given  quantity  of  output  from  the  program.   In 
addition,  if  the  NARF  has  no  control  over  the  prices  it  pays  for  factor 
then  a  cost  function  may  be  derived  from  the  program: 

Min  C  =  CQ  +  P-jZ-l  +  P2Z2  +  P3Z3  (1) 

s.t.   W  =  f  (Zlt  Z2,  Z3) 
,  C  =  CQ  +  g(W,  px,  p2,  p3)  (2) 


as 


In  the  above  program  C„   is  fixed  cost;   P  ,  P  ,  and  P   are  the 
prices  of  factor;  and  W  is  a  measure  of  output  for  the  program. 

It  is  important  to  notice  that  the  costs  of  production  are  thus 
related  to  several  variables  which  influence  the  environment  of  the  organ- 
ization.  The  relation  between  program  costs  and  prices  and  outputs  is 
thus  captured. 

Two  functions  of  the  same  form  as  g  may  be  estimated  by  econometric 
techniques  one  each  for  the  before  WIPICS  program  and  the  after  WIPICS 
program.   Finally,  the  hypothesis  that  these  functions  are  significantly 
different  from  one  another  may  be  tested  for  some  level  of  confidence 
and  we  may  determine  if  the  relation  which  describes  costs  in  the  program, 
has  changed  significantly  with  the  implementation  of  the  WIPICS  system. 

Assuming  the  cost  functions  are  different  from  one  another,  they 
may  be  used  to  describe  changes  in  the  situation  after  the  technological 
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change.   Whether  the  changes  are  beneficial  or  not,  may  depend  on  the 
environment  of  the  enterprise  at  a  particular  point  in  time.   For 
example,  Figure  5  shows  projection  of  two  cost  functions  on  the  cost, 
quantity  axes.   Suppose  function  B  describes  the  before  WIPICS 
situation  and  function  A  the  after  situation.   Then  whether  the  tech- 
nological change  is  beneficial  or  not,  depends  on  the  expected  level 
of  output  of  the  program.   That  is,  the  cost  functions  allow  one  to 
compare  the  before  and  after  situation  at  alternative  levels  of  output 
and  the  conclusions  may  very  well  depend  on  the  level  of  output  chosen. 


Cost 


W  =  output 


FIGURE  5.   Illustrative  Cost  Curves 


This  approach  to  auditing  cost-effectiveness  analyses  of  tech- 
nological change  will  permit  one  to  compare  before  and  after  situations 
even  if  other  things  do  change.   In  particular,  changes  in  output  and 
and  prices  of  factors  are  automatically  accounted  for  in  the  model.   In 
addition  any  action  which  the  manager  of  the  organization  takes  to  mini- 
mize the  cost  of  producing  a  given  level  of  output  is  compatible  with  the 
model  and  consistant  with  a  valid  interpretation  of  the  results. 
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Finally,  anticipated  changes  in  the  constraints  on  the  manager 
or  his  enterprise  may  be  incorporated  into  the  programming  problem 
at   (1)   and  these  changes  also  may  be  handled  if  data  on  the  constraints 
are  collected  and  used  in  the  estimation  procedure. 

Two  alternative  production  functions  have  been  used  in  the  study 
so  far.   The  first  of  these  is  the  Cobb-Douglas  production  function: 


al  &1  Yi 
wl  =  A1Z11Z12Z13 

a2  32  Y2 
W2  =  A2Z21Z22Z23 

Here  W..  and  W  measure  output  of  programs  one  and  two  respectively, 
Solving  the  programming  problems  at  (2)  yields: 

c  =  c0  +  C]_  +  c2 


where  1  al  31         Yl 


a1+g1+Yl    a1+ei+Tl    a1+61+Yl    a^S^ 
Cl  ~  A1W1         Pl  2  P3 

1         a2         62         Y2 


„     *  a2+62+Y2  p  a2+32+Y2  p  a2+32+Y2  _   a2+62+Y2 
C2  "  A2  2  1  2  P3 

and  A-   and  A_   are  constants.   The  properties  of  production  functions 


require  that 


A  >  0,    A2  >  0 


0  <  a1,31,Y1,ct2,32,Y2   <  1 

The  second  production  function  used  in  the  study  is  the  C.  E.  S. 
production  function.   Here  an  additional  specification  is  imposed,  that 
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one  of  the  inputs  is  determined  by  forces  outside  the  organization's 
control  so  that  there  are  only  two  decision  variables  in  the  production 
function.   Here: 


"pl  "Pl  "°l/pl 

Wl  =  V61Z11  +  (1"61)Z12  1 


-P?  -P7  ~ao/Po 

W2  "  V62Z21  +  (1-62)Z22  1 


Now  letting  C'  =  C  +  ?2^13+Z23^      a  programming  problem  similar  to   (2) 
may  be  solved  to  yield. 


where 


C  =  C0  +  Cl  +  C2 


/Zll\  /Wl\°  "pl  1/P1 


C2   =    (P2+P 


.©#'- 


-P2   1/P2 
62+62(Z21/Z22)        ) 


11 
12 


(1- 

6 


1P2      J 


1_ 

Pl+1 
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:22 


(1-62)Pl] 
62P2     J 


P2+l 
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The  properties  of  production  functions  require  that  the  C.E.S.  parameters 
conform  to  the  following  restrictions: 

al»°2»Yl»Y2  >  °'   0<  61'62  <  1' 
and  pl'p2  >  -1' 

It  may  be  noted  in  passing  that  either  production  function  may  be 
used  to  investigate  returns  to  scale.  For  the  Cobb-Douglas  production 
function  decreasing,  constant  or  increasing  returns  to  scale  obtain  as 
a  +  3  +  Y  is  less  than,  equal  to,  or  greater  than  one  respectively. 
For  the  C.E.S.  production  function  decreasing,  constant,  or  increasing 
returns  to  scale  obtain  as  a  is  less  than,  equal  to,  or  greater  than 
one  respectively. 

Finally,  it  may  be  noted  that  the  C.E.S.  function  is  much  more 
general  that  the  Cobb-Douglas  as  it  approaches  the  latter  as  a  special 
case  as  p— ►O. 
C.   THE  CONTINUOUS  PRODUCTION  FUNCTION  FOR  A  JOB 

This  production  model  is  very  similar  to  the  above  model.   It  does, 
however,  overcome  one  troublesome  assumption  above,  namely  the  require- 
ment that  the  output  of  a  program  be  homogeneous.   Here  the  production 
function  is  applied  to  a  job  defined  as  a  particular  type  of  repair  on 
a  particular  airframe  or  engine.   Since  the  time  to  complete  the  same 
job  may  vary  over  the  period  analysed,  the  number  of  days  a  particular 
job  was  in  shop  is  introduced  as  an  explicit  variable  in  this  case. 

These  cost  functions  for  each  job  type  may  be  derived  and  the 
parameters  estimated.   The  functions  may  then  be  estimated  again  for 
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the  after  WIPICS  situation.   The  two  sets  of  functions  may  then  be 
compared  directly  as  above.   Also,  the  total  costs  for  some  representa- 
tive distribution  of  work  load  may  be  predicted  by  each  as  a  further 
test  of  significant  change  between  the  before  and  after  WIPICS  situation. 
This  production  model  for  a  job  enjoys  similar  characteristics  to  the 
production  model  for  a  major  program. 

The  mathematical  model  used  for  the  continuous  job  production  func- 
tion is  also  a  production  function  of  the  Cobb-Douglas  variety.   Here 
however,  the  function  applied  only  to  a  single  job  and  as  a  result  out- 
put was  measured  by  a  constant   (Wjj),   where  the  subscripts  refer  to 
the  type  of  engine  or  airframe  being  reworked  and  the  type  of  repair 
being  done  respectively.   This  has  the  effect  of  including  a  dummy 
variable  in  the  cost  function  associated  with  the  job  and  for  all  prac- 
tical purposes  makes  it  impossible  to  measure  returns  to  scale  for 
either  program.   The  detailed  cost  functions  used  are  discussed  in 
section  four  of  the  report.   The  advantage  of  this  model  is  that  it 
does  not  require  a  homogeneous  output  measure  to  be  defined.   The 
disadvantage  is  that  it  requires  more  data  and  more  judgement  in  deter- 
mining an  appropriate  distribution  of  workload  for  comparison.   In 
addition,  if  a  job  occurs  in  the  before  situation  that  is  not  represented 
in  the  after  situations  or  vice  versa,  the  effect  on  this  job  cannot 
be  compared. 
D.   THE  LINEAR  ECONOMIC  MODEL 

Linear  programming  is  a  mathematical  technique  for  solving  constrained 
optimization  problems  of  a  special  type.   A  linear  economic  model  is  a 
specific  type  of  linear  program  consisting  of  the  maximization  (or 
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minimization)  of  a  linear  function  of  n  variables  subject  to  m  linear 
inequality  constraints.   Linear  programs  (and  their  close  mathematical 
relative  the  input-output  models)  have  become  increasingly  important 
to  microeconomic  theory  in  recent  years. 

In  the  linear  economic  model  being  constructed  it  is  necessary 
to  define  a  process.   A  linear  production  process  is  an  activity  by  which 
one  or  more  outputs  are  produced  in  fixed  proportions  by  the  application 
of  one  or  more  input  factors  in  fixed  proportions.   In  this  case  there 
is  at  least  one  process  per  job  type.   The  production  process  as  defined 
is  homogeneous  of  degree  one  which  implies  constant  returns  to  scale. 
For  example,  if  all  inputs  to  a  process  are  doubled  then  the  output  will 
also  be  doubled.   A  linear  production  function  is  formed  from  a  collec- 
tion of  linear  production  processes  that  may  be  used  simultaneously. 

The  optimal  solution  to  a  linear  economic  model  as  described  above 
consists  of  finding  the  combination  of   m  processes  from  n  available 
processes  such  that  a  linear  objective  function  is  maximized  (or 
minimized) . 

The  following  assumptions  are  made  in  the  formulation  of  a  linear 
economic  model: 

(1)  The  estimated  processes  (reworking  of  engines  and  aircraft) 

are  linear  functions  and  therefore  these  processes  exhibit  constant  returns 
to  scale. 

(2)  The  above  linear  processes  may  be  estimated  by  the  aggregation 
of  a  finite  set  of  observations  over  some  time  period. 

(3)  The  management  objective  of  the  NARF  will  be  assumed  to  be 
minimization  of  costs  subject  to  completion  of  all  work  demanded  by  the 
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operational  forces  of  the  Navy. 

(4)  Prices  used  in  the  model  are  constant  and  may  be  estimated 
from  the  production  data  furnished  by  the  NARF. 

To  present  the  model,  it  is  necessary  to  have  some  mathematical 
notation  available.   Definitions  will  be  as  follows: 


m  = 


n  = 


R  = 


number  of  output  constraints 

number  of  production  processes 

(n  >  m) 

column  vector  of  available  resources 


Rl 
R2 
R3 


P  =  column  vector  of  prices  associated  with  Rl,  R2,  and  R3 
respectively 

=  [zi] 


Z  =  column  vector  of  activity  levels 

i  =  l,...,n 


T  =  Technology  matrix  of  observed  processes  at  NARF  ■ 
[(m+3)  x  n  matrixj 


where 


T   =  units  of  output 
(mxn  matrix) 


1  0  0  0  0  ...0 
0  110  ...0 
0  0  0  10  ...0 


0  .  .  .  .   0  1 


and 


T?  =  resource  matrix 


"  FJ 


i  =  1,  2,  3 
j  =  l,...,n 
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with  r   =  amount  of  i —  resource  used  per  unit 
output  of  j —  activity. 


Y  = 


column  vector  of  production 
output  desired 


M 


i  —  1, . . . ,m. 


T„  =  diagonal  matrix  of  number  of  days  in  shop 
for  each  process  per  unit  of  output 


v  O 

X. 

o  \ 


C   =   column  vector  of  penalty  costs 

i  =  1 , . . . ,n 


[ci] 


where  c .   =  penalty  cost  in  dollars  incurred  for  each  day  in  shop 

1~V» 

associated  with  the  i —  process. 
Now  that  a  basic  mathematical  vocabulary  exists  for  the  variables, 
processes  and  costs,  the  linear  economic  model  will  be  presented  and 
discussed: 
objective  function:  minimize  P^T  Z  +  C  T  Z 


-T, 


-Y 


constraints: 

subject  to: 

The  objective  function  consists  of  two  terms: 

T 
P  'T-^Z  =  actual  cost  in  dollars  of  resources  used  for  activity 

level  vector  Z 

T 
C  •  T_»  Z  =  penalty  cost  in  dollars  for  activity  level  vector  Z. 

The  constraint  set  may  also  be  separated  into  two  parts  for  discussion: 

-T  •  Z  <_ -Y   is  a  set  consisting  of  the  first  m  rows  of 

the  constraints  which  force  the  activity  level  vector 

Z   to  choose  a  set  of  processes  which  satisfy  the 
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production  vector  Y. 

T„«Z  €.    R   consists  of  the  last  3  rows  of  the  constraint  set  and 
ensures  that  actual  resources  used  do  not  exceed 
available  resources. 

It  is  clear  that  the  size  of  the  resource  vector  R  is  highly 
dependent  on  the  production  vector  Y.   In  fact,  these  two  vectors  must 
be  chosen  carefully  to  avoid  an  infeasible  linear  program.   The  easiest 
approach  to  avoiding  an  infeasible  linear  program  is  to  make  the  re- 
source vector  R  very  large,  therefore  the  optimal  solution  to  the 
model  will  not  be  constrained  by  an  active  resource  constraint.   In 
economic  terms,  this  means  that  the  "customer"  is  willing  to  pay  the 
NARF  as  much  as  necessary  to  accomplish  the  work  specified  by  production 
vector  Y.   If  the  NARF  is  assumed  to  be  efficiently  operated,  then 
the  costs  incurred  will  be  minimized. 

In  summary,  the  objective  function  of  the  model  will  be  a  minimum 
when  the  model  is  not  constrained  by  the  resource  vector  R.   In  other 
words,  the  costs  are  minimum  when  the  resource  constraints  are  inactive 
constraints.   The  value  of  the  objective  function  represents  the  total 
cost  incurred  to  accomplish  production  vector  Y  based  on  past  perfor- 
mance data  of  the  NARF. 

It  should  be  noted  that  the  linear  economic  model  presented  in  this 
chapter  is  not  a  production  management  tool  in  the  sense  that  the  model 
chooses  "processes"  by  which  the  NARF  should  rework  aircraft  or  engines. 
The  model  only  provides  a  budget  cost  plus  a  penalty  cost  for  a  specified 
amount  of  work  to  be  done.   Some  information  is  available  from  the  model 
concerning  tradeoff  values  among  the  three  resources.   The  model  can  also 
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determine  the  minimum  amount  of  any  single  resource  required  to  accomplish 
the  specified  work. 

As  with  the  previous  models,  the  linear  economic  model  is  estimated 
from  both  before  and  after  WIPICS  data  and  the  two  resulting  models  may 
be  compared  process  by  process  and  for  a  representative  workload  at  the 
NARF.   The  data  requirements  of  this  production  model  approximate  those 
of  model  two  and  the  estimation  procedure  is  not  as  well  developed  in 
this  area  as  it  is  for  the  previous  two  models.   In  addition,  the  assump- 
tion of  constant  returns  to  scale  and  a  small  number  of  discrete  ways  to 
combine  resources  are  restrictive.   Nevertheless,  the  linear  economic 
model  does  preserve  more  detail  about  the  production  process  at  the  NARF 
and  for  this  reason  it  is  used  and  compared  to  the  previous  models.   It 
also  permits  changes  in  prices,  and  the  level  and  distribution  of  output 
requirements  to  directly  affect  costs  in  a  program. 
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III.   DATA  AND  ITS  USE 
A.   DESCRIPTION  OF  AVAILABLE  DATA 

The  preceding  mathematical  production  models  are  defined  in  terms 
of  physical  measures  of  inputs  and  outputs  and  well  defined  prices  for 
these  quantities.   In  fact,  data  on  these  variables  cannot  be  obtained 
in  most  large  organizations  and  NARFNI  is  no  exception.   The  data  obtained 
from  NARFNI  consists  of  information  on  837  aircraft  and  1865  engine  re- 
works performed  during  the  period  March  1970  to  March  1973.   For  each 
rework  the  following  information  was  available: 

(1)  Type  of  engine  or  aircraft 

(2)  Identification  number 

(3)  Type  of  work  done 

(4)  Induction  date 

(5)  Production  date 

(6)  Production  load  norm  (man-hours)  (NORM) 

(7)  Airframe  change  man-hours 

(8)  Direct  labor  hours  expended  (DMHR) 

(9)  Direct  labor  cost  (DLB$) 

(10)  Direct  material  cost  (DML$) 

(11)  Applied  overhead  cost  (D0H$) 

(12)  Navy  Industrial  Fund  (NIF)  rate  (NIFR) 

The  manner  in  which  the  above  statistics  are  accumulated  and  used  by  NARF 
North  Island  are  enumerated  in  [7] 

In  order  to  keypunch  the  basic  data  it  was  necessary  to  assign  codes 
for  different  types  of  engines/aircraft  and  the  types  of  work  done.   These 
codes  are  listed  in  Tables  I  and  II.   The  coded  data  is  listed  for  reference 
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purposes  in  Spooner  [1]  and  McGarrahan  [6], 

One  additional  item  of  information  concerns  a  penalty  cost  assigned 
to  aircraft  and  engine  down  time  at  the  NARF.   These  penalty  costs  per 
day  were  calculated  from  average  flyaway  unit  procurement  costs  for  each 
engine  and  aircraft.   The  particular  penalty  costs  used  are  reported  by 
McGarrahan  [6], 

These  data  were  used  to  form  several  proxies  for  the  variables  of 
Section  Two.   A  brief  description  of  each  follows: 
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TABLE  I 


Codes  Assigned  to  the  Engine  Program 


Engine  Type 

Code 

T58-GE-1  (A/F) 

51 

T58-GE-3  (A/F) 

53 

T58-GE-5  (A/F) 

55 

T56-A-8P 

56 

T58-GE-5  (C/G) 

57 

Engine  Type 

Code 

T64-GE-3  (A/F) 

63 

T64-GE-6B 

65 

T64-GE-6B  (PAWN) 

66 

T64-GE-7  (A/F) 

67 

T64-GE-413 

69 

J57-P-4A/22 

J57-P-10 

J57-P-20A 

J57-P-22 

J57-P-420 


Work  Type* 

OVHL 

OVHL/CONV 

PAR/REP 

SUP /REP 

SUP/REP/CONV 


71 
72 
73 
74 
75 


J79-GE-8B 

91 

J79-GE-8B/C 

92 

J79-GE-8B  (RDTE) 

J79-GE-10 

95 

Code 
01 
02 
03 
04 
05 


T58-GE-8F 
T58-GE-8B 
T58-GE-8B/F 
T58-GE-8B  (C/G) 
T58-GE-8B  (HH2) 
T58-GE-8B/F  (CONV) 
T58-GE-10 


81 
82 
83 
84 
85 
86 
89 


*  Abbreviations  are  defined 


as  £ 

ollows 

OVHL 

= 

overhaul 

CONV 

= 

conversion 

PAR 

■ 

Planned  aircraft 
repair 

SUP 

ca 

Supply 

REP 

= 

Repair 

SEA 

= 

South  East  Asia 

CONUS 

= 

Continental  United 
States 
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TABLE  II 
Codes  Assigned  to  the  Aircraft  Program 


Aircraft  Type  Code 

C-2A  10 

E-2A/B  11 

F-4J  21 

F-4B  22 

F-4G/B  23 

F-8J  25 

F-8H  26 

RF-8G  27 


Work  Type*         Code 
OVHL  01 

PAR  02 


PAR/CONV 

03 

PAR/MOD 

04 

PAR/MOD/REP 

05 

PAR/REP 

06 

PAR/ SEA 

07 

PAR/CONUS 

08 

Aircraft  Type 

Code 

Ch-3B 

31 

RH-3A 

32 

SH-3A 

33 

SH-3A/G 

34 

SH-3D 

35 

CH-46A 

41 

CH-46D 

42 

CH-46F 

43 

UH-46A 

44 

UH-46D 

45 

CH-53A 

48 

CH-53D 

49 

*See  Table  I  for  Work  Type  abbreviations, 
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1.  Output.   Physical  measures  of  output  of  the  NARF  would  require 
hundreds  of  variables  on  numbers  of  aircraft  and  engines  of  various  types 
with  several  different  kinds  of  repair  work  done.   Furthermore,  there  is 
no  obvious  common  denominator  of  the  repair  work.   So  many  different 
measures  of  output  would  be  extremely  difficult  to  handle  and  would 
require  many  more  observations  to  produce  statistical  significance. 
Hence,  a  proxy  variable  of  production  load  norm,   N,   was  chosen  to 
measure  output.   Production  load  norm  has  the  advantage  that  across  jobs 
it  is  measured  in  the  same  units,  expected  man-hours  required  to  complete 
the  job.   In  addition,  it  is,  in  many  cases,  what  determines  the  cost  to 
the  fleet  of  having  the  repairs  made.   That  is,  the  fixed  price  of  a 

job  is  negotiated  as  two  components,  the  production  load  norm,  as  a 
measure  of  the  difficulty  of  the  job,  and  the  expected  NIF  rate,  as  the 
expected  cost  per  man  hour  on  the  job.   These  might  usefully  be  thought 
of  as  measures  of  the  quantity  of  repairs  to  be  completed  and  the  price 
per  unit  of  the  repairs. 

2.  Input  one  -  Labor.   Direct  man-hours  expended,   L,   are  used 
as  a  measure  of  labor  used.   This  variable  excludes  indirect  man-hours 
and  fails  to  distinguish  among  alternative  types  of  labor  used.   In 
spite  of  these  drawbacks,  it  is  the  most  accurate  of  the  proxy  variables. 

3.  Price  of  input  one  -  Wage  rate.   The  ratio  of  direct  labor 
costs,   L$,   to  direct  man-hours  expended,   L,   is  used  as  a  proxy  for 
the  wage  rate  P  .   Because  this  variable  is  averaged  over  all  types  of 
labor  it  may  be  influenced  by  factors  other  than  a  general  increase 

in  wages.   For  example,  an  airframe  that  requires  very  technical  work 
and  uses  high  priced  labor  exclusively  will  be  associated  with  higher 
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values  of  this  proxy.   Also,  periods  of  time  that  involve  large  quantities 
of  overtime  labor  will  show  high  values  of  the  variable.   Still,  this 
variable  is  thought  to  be  a  reasonably  accurate  measure  of  the  wage  rate. 

4.  Input  two  -  Items  to  be  repaired.   It  seems  clear  that  if  out- 
put is  to  be  a  measure  of  repairs  on  engines  or  airframes  then  one  input 
must  be  repairable  engines  or  airframes.   Unfortunately,  these  items 
have  many  different  descriptions  and  are  in  various  states  of  disrepair. 
As  a  crude  proxy  for  this  variable  the  unweighted  number  of  items  in 
ship  at  any  point  in  time,   I,   is  used  as  a  measure  of  items  to  be 
repaired. 

5.  Price  of  input  two  -  Penalty  cost  per  item.   The  NARF  does  not 
pay  for  items  to  be  repaired  nor  does  it  provide  "loaners"  to  the 
fleet  while  the  repairs  are  being  undertaken.   Nevertheless,  from  the 
point  of  view  of  the  Navy  this  pipeline  of  repairable  items  is  costly 
and  both  the  NARF  and  the  Navy  are  concerned  with  reducing  the  size  of 
the  pipeline.   Hence,  the  models  that  we  are  using  include  as  costs  of 
the  NARF  a  penalty  cost  for  each  repairable  item  held  by  the  NARF  on  any 
day.   Our  proxy  measure  of  penalty  cost  per  item  per  day  is  procurement 
cost  of  the  item  divided  by  expected  service  life,  P  .   This  measure 
tends  to  overstate  the  true  costs  because  it  ignores  scrap  value  of  the 
item.   However,  this  is  more  than  compensated  for  by  its  understatement 
of  cost  due  to  ignoring  the  time  value  of  money.   The  particular  penalty 
costs  used  are  listed  in  McGarrahan  [6], 

6.  Input  three  -  Material.   Because  many  small  items  of  material 
and  parts  are  consumed  no  attempt  has  been  made  to  construct  a  quantity 
index  of  material.   Instead,  the  dollar  value  of  material  consumed,  M$, 
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is  used  as  a  proxy  for  this  variable. 

1  -      Price  of  input  three.   Since  the  quantity  of  material  is 
measured  in  dollars  the  price  of  material  is  defined  to  be  one.   Over 
time  periods  long  enough  for  inflation  to  be  a  problem  an  appropriate 
price  index  would  be  used  for  the  price  of  material. 

In  addition  to  the  definition  of  proxy  variables,  one  further 
adjustment  was  required  to  use  the  available  data  from  NARF  NORIS  to 
estimate  production  and  cost  functions.   The  available  data  consisted 
of  observations  on  expenditures  for  each  job  with  different  jobs  lasting 
varying  lengths  of  time.   On  the  other  hand,  the  continuous  program 
production  functions  describe  the  entire  program  over  some  given  length 
of  time.   Thus,  the  data  were  aggregated  in  the  following  manner:   For 
each  job,  the  values  of  norm,  man-hours,  labor  cost,  material  cost,  over- 
head cost  and  penalty  cost  were  prorated  equally  over  the  days  for  which 
that  job  was  in  the  shop.   Then,  for  each  day  in  the  period  under  con- 
sideration, the  prorated  values  were  totalled  for  all  jobs  which  were 
in  the  shop  on  that  day.   The  result  of  this  aggregation  is  that  for  each 
day  in  the  period  under  consideration,  we  have  an  estimate  of  the  total 
number  of  man  hours   (L)   used  on  that  day,  the  total  costs  of  labor 
(L$)   for  that  day,  the  total  cost  of  materials   (M$)   for  that  day,  the 
total  cost  of  overhead   (0$)   for  that  day,  the  total  penalty  cost   (P$) 
incurred  on  that  day,  and  the  total  hours  of  norm   (N)   produced  on  that 
day.   In  addition,  the  number  of  jobs  in  the  shop   (I)   on  each  day  was 
tabulated.   All  of  the  prorating  and  aggregating  was  done  separately 
for  the  two  programs — aircraft  and  engines.   A  computer  program  to  per- 
form this  task  is  included  in  Spooner  [1]  . 
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These  aggregated  data  were  used  as  described  in  section  A  to  estimate 
the  parameters  of  the  cost  and  production  models. 
B.   PRELIMINARY  DATA  ANALYSIS 

As  a  first  step  in  determining  the  relationships  among  the  raw  data 
elements,  the  matrix  of  correlation  coefficients  was  calculated  for  each 
program.   These  results  for  the  engine  and  airframe  programs  are 
reported  in  Table  III  and  IV,  respectively. 

It  is  perhaps  important  to  note  the  relatively  low  correlation  be- 
tween direct  material  cost  and  any  of  the  other  variables  in  both  programs. 
This  is  one  possible  indication  of  the  fact  that  material  costs  are  thought 
to  be  the  least  reliable  of  the  data  obtained.   It  is  also  interesting 
to  note  the  relation  of  number  of  days  in  shop  to  the  other  variables. 
In  the  shorter  engine  program,  these  correlations  are  relatively  low 
while  they  are  significantly  higher  for  the  longer  aircraft  program. 

In  addition,  the  aggregated  data  was  used  to  get  a  preliminary  view 

of  the  flow  of  work  over  time  at  the  NARF.   Conversations  with  NARFNI 

personnel  indicated  that  it  was  their  opinion  that  induction  rates  were 

fairly  constant,  independent  of  time,  while  production  rates  were  low 

at  the  beginning  of  a  quarter  and  high  at  the  end  of  a  quarter.   The 

average  daily  induction  and  production  rates  were  computed,  using  a 

moving  block  average. 

These  rates  were  computed  from  both  the  aircraft  and  engine  programs 

in  terms  of  hours  of  norm  arriving  or  leaving  per  day,  actual  man  hours 
of  work  arriving  or  leaving  per  day,  and  actual  physical  units  arriving 
or  leaving  per  day.   (Figure  6  and  Figure  7).   The  results  were  not  sen- 
sitive to  changes  in  the  size  of  the  moving  window  in  the  range  of  seven 
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ENGINE  INDUCTION  AND  PRODUCTION  RATES 
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AIRCRAFT  INDUCTION  AND  PRODUCTION  RATES 
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to  twenty-one  days.   These  plots  tended  to  confirm  NARFNI  personnel  in 
their  beliefs  about  the  production  and  induction  rates  over  time.   Pro- 
duction did  increase  toward  the  end  of  a  quarter  and  did  drop  off  sharply 
at  the  beginning  of  a  quarter.   The  induction  rate  did  have  some  varia- 
tion over  time,  but  it  was  about  one-fifth  as  great  as  the  variation  of 
the  production  rate. 

Number  of  jobs  in  shop  versus  time  was  plotted  to  confirm  this 
relationship  between  induction  rate  and  production  rate.   Number  of  jobs 
in  shop  should  be  increasing  at  the  beginning  of  a  quarter  when  produc- 
tion rate  is  less  than  induction  rate  and  should  be  decreasing  at  the 
end  of  the  quarter  when  production  rate  is  greater  than  induction  rate 
(Figure  8).   This  plot  of  jobs  in  shop  versus  time,  with  a  maximum  near 
mid-quarter,  is  consistent  with  the  production  rate  and  induction  rate 
relationship  described  above. 

The  variable,  number  of  jobs  in  shop   (I),   was  considered  to  be 
descriptive  of  the  production  and  induction  rates  and,  incidentally, 
also  descriptive  of  shop  congestion.   By  using  number  of  jobs  in  shop 
as  an  independent  variable  in  the  Cobb-Douglas  production  function,  it 
was  possible  at  once  to  describe  efficiencies  due  to  specialization  of 
personnel,  inefficiencies  due  to  shop  congestion,  and  work  flow  through 
NARFNI. 
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IV  PROCEDURES  FOR  ESTIMATION  OF 
MODEL  PARAMETERS 

A.   THE  LINEAR  ECONOMIC  MODEL 

To  formulate  a  linear  economic  model  it  is  necessary  to  define 
a  process.   A  process  consists  of  a  mathematical  description  of  the 
amount  of  resources  used  (i.e.  labor,  material,  etc.)  to  produce  a  unit 
amount  of  an  output  measure.   The  problem  is  how  are  the  raw  data  obser- 
vations used  to  estimate  a  process.   The  most  natural  way  would  be  to 
estimate  a  process  as  an  average  amount  of  resources  required  to  do  a 
specific  type  of  work  on  a  certain  engine  or  aircraft.   This  procedure 
may  not  be  desirable  because  there  may  be  alternate  processes  available 
to  produce  the  same  output.   For  example,  to  overhaul  an  F-8J  it  may 
be  possible  to  use  a  "normal"  amount  of  man-hours  and  have  a  low  material 
cost.   An  alternate  process  may  exist  whereby  the  same  work  may  be 
accomplished  by  spending  more  money  on  material  and  reducing  the  man- 
hours  required.   The  essence  of  a  linear  economic  model  is,  then,  to 
identify  these  alternate  processes  and  to  use  linear  programming  to 
select  the  most  efficient  processes  to  operate. 

In  an  attempt  to  identify  these  alternate  methods  within  the  basic 
data  observations  a  clustering  algorithm  was  developed  based  on  the 
Hierarchical  Clustering  Strategy  of  Lance  and  Williams  [8].  The  general 
procedure  is  to  select  a  group  of  observations  containing  several  different 
types  of  engines  (or  aircraft)  and  some  different  types  of  work  within  each 
engine  group.   These  observations  are  then  analyzed  using  the  clustering 
algorithm  for  similarity  of  the  vector  of  input/output  measures.   Vectors 
which  are  similar  will  use  input  resources  in  approximately  the  same 
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fixed  proportion  for  a  unit  output  measure.   Similar  observations  can 
then  be  averaged  to  yield  a  "process". 

Several  trial  selections  were  made  with  both  engine  and  aircraft 
data.   The  engine  data  was  observed  to  cluster  groups  of  the  same  engine 
type  together  with  a  high  degree  of  consistency.   For  example,  if  20  J-79 
engines  were  selected  as  part  of  a  trial  run  of  100  observations,  a 
typical  result  would  be  for  17  or  18  of  the  original  20  to  be  clustered 
in  the  same  group.   However,  the  trial  selection  with  the  aircraft  data 
were  not  nearly  so  consistent.   The  cluster  analysis  showed  the  aircraft 
data  had  wide  variations  among  the  input  variables.   The  clusterings 
observed  were  highly  irregular  and  contained  a  mixture  of  different 
aircraft  and  different  work.   Since  the  aircraft  clusters  which  resulted 
from  this  preliminary  analysis  were  not  easily  interpretable  as  "processes", 
it  was  necessary  to  do  some  preliminary  classification  before  using  the 
clustering  algorithm.   In  particular,  it  seemed  ridiculous  to  have  processes 
which  involved  radically  different  job  types  even  if  they  seemed  to  use 
resources  in  the  same  proportion,  so  a  decision  was  made  to  form  processes 
consisting  of  only  similar  engines  (or  similar  aircraft)  with  identical 
types  of  work. 

For  the  engine  data,  similar  engine/work  types  were  grouped  together 

within  each  job  order.   The  resulting  raw  data  deck  was  then  ordered  by 

engine  type,  job  type  and  calendar  quarter  in  which  the  work  was  done 

(in  special  cases  where  a  very  few  observations  were  available  the  engine/ 

job  types  were  not  separated  into  calendar  quarters).   The  average  of  the 

observations  in  each  calendar  quarter  then  provided  an  aggregated  obser- 
vation on  each  enging/'work  type  worked  on  by  the  NARF  during  that  quarter. 
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This  observation  was  taken  to  be  a  potential  process  for  that  engine/ 
work  type.   At  this  point  there  were  47  different  engine/work  types 
and  103  processes  by  which  the  work  could  be  done.   A  computer  program 
was  used  to  examine  the  dominance  relationships  among  all  processes  with 
the  same  engine/work  type.   This  computer  program  is  described  in  Appen- 
dix G  of  Myers  [2],   The  dominance  relationships  among  processes  are  very 
important  in  a  linear  program.   For  example,  if  process  1  and  process 
2  produce  the  same  output  but  process  2  uses  more  of  every  required 
resource  than  process  1,  then  process  1  dominates  process  2.   In  a  linear 
program  that  minimizes  costs,  process  1  will  always  be  selected  as  pre- 
ferred to  process  2.   Therefore,  if  the  linear  programming  model  is  to 
have  alternate  processes  that  reflect  choices  to  be  made  then  the  domi- 
nance must  be  eliminated  between  all  processes  with  the  same  output. 

The  dominance  program  was  run  with  the  103  processes  as  previously 
determined  based  on  job  order  numbers.   The  dominance  relationship 
between  each  pair  of  processes  that  produced  the  same  output  was  examined. 
If  no  dominance  existed  then  that  group  of  alternate  processes  was  left 
alone.   If  dominance  existed  among  any  of  the  processes  in  the  group  then 
a  subjective  decision  was  made  to  combine  two  of  the  processes.   For 
example,  if  there  were  five  processes  and  dominance  existed  between  three 
pairs  of  them,  then  two  processes  would  be  selected  to  be  combined.   The 
decision  on  which  pair  to  combine  was  entirely  subjective.  The  group 
would  then  have  four  alternate  processes  remaining.   The  dominance  com- 
puter program  would  then  be  run  again  and  the  remaining  alternate  processes 
would  be  examined  for  dominance.   This  procedure  was  repeated  until  all 
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dominance  was  eliminated  within  process  groups  that  had  the  same  output 
(same  engine/work  type).   Since  the  outputs  were  intended  to  be  measured 
separately  (as  opposed  to  an  aggregate  output  measure)  it  was  not  ne- 
cessary to  examine  the  dominance  between  processes  of  different  outputs. 
The  original  103  potential  processes  were  aggregated  down  to  81  processes 
(on  the  47  engine/work  type)  by  this  dominance  removal  procedure.   The 
final  81  processes  were  used  as  input  for  the  Linear  Economic  Model  on 
engines  described  earlier. 

The  aircraft  data  consisted  of  365  raw  data  observations.   Con- 
sidering each  type  of  aircraft  and  type  of  work  as  a  separate  output 
meant  that  70  outputs  were  required.   This  small  amount  of  data  then 
had  to  be  used  to  estimate  not  less  than  70  distinct  processes.   A 
decision  was  made  to  estimate  only  a  single  process  for  each  output  be- 
cause of  the  limited  data  available.   In  this  case  there  were  no  alternate 
processes  so  it  was  not  necessary  to  check  for  dominance.   The  linear 
program  in  this  case  would  not  have  a  choice  among  processes  and  there- 
fore the  solution  becomes  trivial. 

An  important  extension  of  this  work  would  be  to  refine  methods  for 
identifying  alternate  processes  in  such  small  data  bases.   Computer 
programs  for  the  data  analyses  described  above  can  be  found  in  appendices 
B  -  G  of  Myers  [2],   The  resulting  linear  programs  for  the  engine  and 
aircraft  model  will  be  compared  to  other  models  and  across  the  Before 
WIPICS  -  After  WTPICS  period  in  Section  VI. 
B.   THE  CONTINUOUS  PRODUCTION  FUNCTION  FOR  A  PROGRAM 

Three  alternative  econometric  models  based  on  the  two  production 
functions  detailed  in  section  II  are  derived  in  this  section.   Each 
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involves  different  specifications  about  how  random  variables  enter  the 
cost  and  production  function  and  hence  different  assumptions  concerning 
the  behavior  of  decision  makers  at  NARF  North  Island.   Throughout  this 
section  the  subscripts  referring  to  the  engine  and  airframe  programs 
have  been  dropped.  Each  model  is  applied  to  both  these  programs. 

Model  one.   The  first  and  simplest  of  the  three  models  assumes  that 
the  relation  among  inputs  and  output  at  the  NARF  is  of  the  form  of  the 
Cobb-Douglas  production  function.   Additionally,  it  is  assumed  that  an 
unobservable  error  term  enters  the  relation  in  a  multiplicative  fashion 
so  that  the  production  function  itself  is  stochastic.   Substituting  the 
proxy  variables  and  including  e    as  the  random  error  yields: 

N.  =  A  L?  TB  M$T  e£i 
1      i  i   i 

where  i  indicates  the  time  period  of  the  observation.   Taking  natural 

logarithms  of  both  sides  yields: 

in    (N  )  =  in   A  +  a  in   L  +  g  £n  I  +  y   in   M$  +  e 

This  equation  is  the  econometric  model  associated  with  model  one  and  is 
estimated  by  linear  regression  techniques.  The  results  are  reported  in 
section  V. 

Model  two.   The  second  model  also  makes  use  of  the  Cobb-Douglas 
production  function.   Here,  however  the  production  function  itself  is 
assumed  to  hold  with  certainty.   Random  errors  enter  the  model  in  the 
process  of  solving  the  cost  minimization  programming  problem.   Since 
the  exact  form  of  the  production  function  is  not  known  the  planned 
quantities  of  inputs  required  to  produce  a  given  quantity  of  output  may 
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linearity  of  the  C.E.S.  cost  function  it  is  estimated  in  two  stages 
instead  of  one.   The  first  stage  involves  the  choice  of  the  relative 
quantities  of  the  two  inputs,  labor  and  the  number  of  items  in  shop. 
This  choice  is  thought  to  be  subject  to  a  random  error  and  is  determined 
by: 

Taking  natural  logarithms  of  both  sides  yields: 


y.  -  60  +  BlXu  +  e± 


where 


y.  -  *n  (L±/I±)  6o  ,  ZL.  ln  i=i 

61  "  ^T  Xli  =  te(PU/PIi  > 


This  equation  is  the  first  stage  of  the  econometric  model  associated  with 
model  three  and  is  estimated  by  linear  regression  techniques. 

Let  the  estimates  derived  for  parameters  in  the  first  stage  be  b~ 

and  b1   respectively  so  that  y.  =  bQ  +  b,X    is  a  prediction  of  y  . 
Also  estimates  of   6   and  p  may  be  derived  as: 


6  = 


eb0/bi  +  l 


and 


.   -1  -  b1 

P  - 


bl 


y     bn+b  X.     L  "I  ("L  1 

Finally  e=e  i     isa  Prediction  of    I   '   Substituting 
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differ  from  those  actually  required.   This  leads  to  build  ups  or  short- 
ages in  inventories  of  inputs  and  thus  to  random  errors  in  the  cost 
function.   See  Trafton  [4,  pp  19-20]  and  Dhrymes  [9,  pp.  232-234]  for  a 
complete  derivation  of  this  model. 

Substituting  the  proxy  variables  and  including  the  random  error 
term  yields: 


C   =  A*N1/a+6+Y  Pa/a+S+Y  pB/a+B+Y   i 
i      i       I  Ii 


where 


C   =  L$  +  M$ .  +  P$ 
Taking  natural  logarithms  of  both  sides  yields: 


in   Ci  =  BQ  +  ^  in   N±  +  $2   Jin  P£i  +  g3  in   P   +  e 


where 

6X  =  l/a+6+Y         3Q  =  in   A* 
B2  =  a/a+3+Y 

33  =  B/a+3+Y 

This  equation  is  the  econometric  model  associated  with  model  two  and  is 
estimated  by  linear  regression  techniques.  The  results  are  reported  in 
section  V. 

Model  three.   Model  three  differs  from  the  previous  two  models  in 
two  regards.   It  is  based  on  the  C.E.S.  production  function  instead  of 
the  Cobb-Douglas  function  but  it  is  also  different  in  that  material  cost 
is  no  longer  an  input  to  the  production  process.   Because  of  the  non- 
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these  estimates  into  the  C.E.S.  cost  function  and  supplying  an  error 


term  yields: 


Cl  '  »!!«« 


u 


N. 


Y 


1 

a 


bo/bi  :  ; 

e      +  1 


+ 


Vbi  .  . 

e      +1 


At 

Li 

l+b, 
bl 

1/p 


where 


C.  =  L$  +  P$ 
i     i 


This  equation  may  be  reexpressed  as: 


yi  =   30  +  Bl   *n  Ni  +  62Xi  +  £i 


where 


I  -  «n<°i>  -  ^Pn +  Pu  [yJI  i 


I     _ 


0 


£n  y 

a 


Bl  "  a 
32  "  p 


X.    =  An 


Vbl 
e  + 


lJ 


1+b, 


I 


Vbi  . . 

e  +1 


giving  the  second  stage  of  the  econometric  model  associated  with  model 
three  and  it  is  also  estimated  by  linear  regression.   The  results  of 
model  three  are  also  reported  below. 
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C.   NONLINEAR  PROGRAMMING  FOR  C.E.S.  ESTIMATION 

Because  the  two  stage  estimation  procedure  for  the  C.E.S.  function 
described  above  produces  parameter  estimates  whose  properties  are  not 
known,  it  was  considered  desirable  to  investigate  alternate  estimation 
strategies.   In  particular,  any  least  squares  parameter  estimation  pro- 
blem can  be  formulated  as  a  nonlinear  optimization  problem.   If  the 
model  is   y  =  f(X,  3)   with  independent  variables  X,   dependent  variable 
y,   and  parameters   6   to  be  estimated,  from  the  n  observations 

(X.,  y.)   i  =  1,...,  n,    then  the  least  squares  solution  is   3  which 

solves  the  NLP. 

n  „ 

min  I      (y  -  f  (X  ,  3)) 

f»  1=1 

If   f(X,  3)   is  linear  in   3,   then  this  is  an  unconstrained  quadratic 
optimization  for  which  standard  regression  produces  a  closed  form  solution. 
Otherwise,  as  in  the  case  of  the  C.E.S.  function,  iterative  nonlinear 
programming  methods  for  gradually  approaching  the  solution  can  be  used. 

Since  the  two  stage  estimation  procedure  sometimes  yields  parameters 
which  have  intuitively  ridiculous  signs,  it  may  be  desirable  to  also 
incorporate  constraints   (such  as  3.  >  0)   into  the  nonlinear  program. 
This  approach  to  estimating  C.E.S.  cost  function  coefficients  was  in- 
vestigated in  some  detail.   The  C.E.S.  cost  function  involves  the 
model 

c  =  Nl/o  y-l/o  T(1  _  5)  +  6  (I/L)-Pj  1/P[PL  +  pi  (i/L)]  e 
where  economic  theory  imposes  the  following  restrictions  on  the  parameters 

a>0,  y  >0>  P  >-l,  0<  6<  1 
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By  transforming  the  variables  we  can  incorporate  these  constraints  into 

2        2        2 
the  optimization  problem.   If  we  let  a  ■  z-  ,  y  »  z_   p  «  z_  -1, 

2 
6  =  (sin  z.)    then  for  any  value  of  the  parameters  z   i=l,  2,  3,  4, 

the  parameters  a,  y»  P  anc*  <$  will  satisfy  the  above.  Hence  the  least 
squares  optimization  problem  can  be  formulated  as  the  unconstrained 
minimization  of  a  function  of  the  four  parameters  z.  ,  z~,  z_,  z,. 
This  minimization  problem  was  investigated  for  the  before  WIPICS  air- 
craft program.   400  before  WIPICS  observations  (Julian  dates  0191  -  1225) 
were  chosen  as  test  data.   Tye  [5]  reports  the  results  of  the  two  stage 
C.E.S.  estimation  procedure  applied  to  the  same  data.   The  400  observa- 
tions were  split  into  two  subgroups  consisting  of  the  first  300  and  last 
100  observations  so  that  the  stability  of  the  cost  function  could  also  be 
investigated.   The  unconstrained  minimization  of  the  least  squares  func- 
tion was  accomplished  by  the  Conjugate  Gradient  method  due  to  Fletcher 
and  Reeves  [10],   The  minimizations  were  performed  for  300,  100  and  all 
400  observations.   In  each  case  several  different  starting  points  were 
chosen  including  the  parameter  estimate  from  the  2-stage  procedure,  to 
test  for  unimodality  of  the  least  squares  objective  function.   The 
results  of  this  procedure  were  not  promising  and  hence  will  not  be  re- 
ported here  in  detail.   The  following  qualitative  remarks  will  indicate 
some  of  the  problems  encountered: 

1.   Numerical  difficulties  were  severe  for  the  highly  nonlinear 
C.E.S.  cost  function.  Typical  runs  started  with  the  least  squares  func- 
tion value  (sum  of  squared  residuals)  from  10   to  10   .   Final  values 

9 

when  the  minimization  terminated  were  on  the  order  of  10  .   Gradient 

components  were  also  frequently  extremely  large  in  magnitude.   As  a 
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result  abnormal  terminations  due  to  numerical  problems  such  as  computer 
overflow  or  underflow  were  encountered  even  in  double  precision.   Careful 
scaling  of  the  variables  was  necessary  to  get  any  results  at  all. 

2.  The  two  stage  C.E.S.  estimates  were  not  anywhere  close  to  least 

30 
squares  solutions,  having  objectives  in  the  range  10   .   When  started 

at  these  estimates,  the  nonlinear  programming  procedure  reduced  the 

9 
objective  quickly  to  about  10  . 

3.  The  solutions  obtained  when  the  nonlinear  minimization  was  started 
from  different  initial  estimates  had  objective  function  values  which 

were  similar,  but  the  parameter  estimates  obtained,  displayed  substantial 
variability.   This  indicates  that  there  are  many  alternative  parameter 
estimates  all  of  which  are  approximately  equivalent  under  the  least  squares 
criterion.   We  do  not  have  other  theoretical  properties  for  these  esti- 
mates (in  particular  their  statistical  properties  are  unknown)  so  the 
choice  of  which  parameter  estimates  to  use  is  not  simple. 

Due  to  these  difficulties,  it  was  decided  not  to  pursue  this  line 
of  investigation  farther.   The  two  estimation  methods  tried  (2  stage, 
nonlinear  programming)  for  the  C.E.S.  cost  function  gave  substantially 
different  results  and  neither  has  known  properties.   Thus,  for  the  bulk 
of  our  work,  the  simpler  Cobb-Douglas  Production  and  Cost  Functions  will 
be  used.   The  Cobb-Douglas  functions  are  linear  in  the  parameters  and 
hence  ordinary  regression  gives  parameter  estimates  with  known  statistical 
properties. 
D.   THE  CONTINUOUS  COST  FUNCTION  FOR  A  JOB 

The  econometric  model  corresponding  to  the  production  model  for  a 
job  is  similiar  to  model  two  described  above.   However,  to  facilitate 
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understanding  of  this  model  a  complete  deviation  is  presented  here. 
Suppose  the  production  function  for  a  job  of  type  i  j   is  of  the  Cobb- 
Douglas  form  with  four  inputs,  i.e. 

a    3     y      6 

W.  ..  =  A.  .  L.  ,.  T,  ..  M$...  NIS,  ., 
ijk    ij   ijk  ijk   ijk    ijk 

here  W.    is  a  measure  of  repairs  produced  by  completing  job  i,  j,  k, 
1 J  k 

T...   is  the  length  of  time  this  job  is  in  shop,  and  NIS,.,   is  the 
ijk  °  J  r'         ijk 

average  number  of  jobs  in  the  shop  while  this  job  is  being  worked  on. 
NIS...  is  assumed  not  to  be  under  the  control  of  the  NARF  for  any  par- 
ticular job  and  hence  is  not  a  decision  variable  in  the  following  costs 
minimization.   Also,  suppose  that  the  NARF  attempts  to  minimize  the  cost 
of  producing  any  given  job  so  that  its  goal  is  to 


Mln  Cljk  "  C0iJk  +  P«Jk  Lijk  +  PIi  Tijk  +  M$ijk 

a    3     Y      <$ 

s.t.   W...  =  A,.  L...  T...  M$...  NIS... 
ijk    ij   ijk  ijk   ijk    ijk 


As  in  model  two  errors  are  expected  in  the  attempt  to  perfectly  meet 
this  goal  and  this  results  in  a  stochastic  cost  function  of  the  form 

1        a        3_       6 

Uv         n-t-A-X-v 

eijk 


A   a+3+Y    a+3+Y    a+3+Y     a+3+Y 


C..,  -  Cn...  =  A,.  W...     P....     PT.      NIS...    e 
ijk    Oijk    ij   ijk     fcijk     Ii        ijk 

Two  alternative  econometric  models  are  formulated  from  the  above 
equation.   The  first  of  these  models  is  based  on  the  assumption  that  out- 
put depends  merely  on  the  type  of  aircraft  being  repaired  and  the  classifi- 
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cation  of  repairs.   Thus  two  par  reworks  on  an  F4-J  would  be  assigned  the 
same  measure  of  output.   Thus 


W...  =  W,  .    (K  =  1,  ....  n) 
ijk    ij 

In  this  case,  the  variable  W. .   is  indistinguishable  from  the  constant 

* 
A..   and  also  the  variable  PT .   is  unable  to  be  distinguished  from  either 

of  the  others.   Hence,  all  three  variables  must  be  combined  into  dummy 

variables  that  vary  only  with  job  description. 

The  first  econometric  model  is  based  therefore  on 


IX 


a+B+Y    a+B+y   _,,k 

C.  ..  -  C_.  ..  =  B,.  Pnj  ..    NIS       e  J 
ijk    Oijk    ij   Aijk      ijk 


or  taking  natural  logarithms  and  redefining  variables 

y...  =  £n  B..  +  n  £n  P....  +  v  £n  NIS...  +  e .  .. 
'ijk      ij        A  ijk  ijk    ijk 


where 


y.  ,.  =  Jin  (C    -  C_.  ..  ) 
^ijk        ijk    Oijk 


1      B 

B.  . 
ij 

A   ^a+B+Y   a+B+Y 

-  A..  W. .     PT. 
ij   iJ      Ii 

H  =  a/a+B+Y 

v  =  6 /a+B+Y 

Another  model  formed  from  the  above  equation  is  due  to  a  different  measure 
of  output.   Here  output  is  supposed  to  be  proportinate  to  production  load 
norm  but  the  factor  of  proportionality  depends  on  the  job  description  thus 
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Wijk  =  aij  Nijk 


In  this  case  also  other  variables  must  be  combined  with  the  constant  term 
to  form  a  viable  model  and  the  resulting  form  of  the  model  is 


^ijk  "  ta  DiJ  +  «1  ^OV  +  62  tn  ^iljk)  +  Eijk 


where 


yijk  =  to  (Cijk  "  CQ±)*> 


B 


*     a+B+Y   a-|^+Y 

D.  .  =  A.  .,  a.  .     P 
ij    13  k  ij      Ii 


6   =  1/a+B+Y 


62  =  a/a+B+y 

The  results  of  some  comparisons  using  this  kind  of  cost  function  for 
a  job  are  reported  in  section  VI  below.   An  alternate  derivation  of  some 
similar  models  is  given  by  McGarrahan  [6]. 
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V   ESTIMATION  AND  TESTING  OF  THE  CONTINUOUS  MODELS 

A.   PROCEDURE 

This  section  reports  the  results  of  applying  regression  analysts 
to  estimate  the  coefficients  of  the  three  continous  models  described  in 
Chapter  IV. 

1.   Cobb-Douglas  Production  Function 

log  N  =  6Q  +  61  log  L  +  B2  log  M$  +  B3  log  I  +  e 


2.   Cobb-Douglas  Cost  Function 


log  C  =  BQ  +  3]_  log  N  +  B2  log  P£  +  83  log  P].  +  e 


where 


C  =  total  cost  =  L$  +  M$  +  P$ 


P  =  price  of  labor  =  L$/L 


P  =  penalty  price  of  items  in  shop  =  P$/l 


3.   C.E.S.  Cost  Function 


£n(L/I)    =   BQ  +   B 


>•■  {V} 


for   stage  one,    and 


Jin  C 


-   ta   {Pj  +  Pn   (£)}-   *'o+  6i   toN+   B2   ia 


l-bx. 


y^(k) 


+  e 


for   stage  2. 
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Recall  that  for  the  C.  E.  S.  cost  function,  total  cost  C  does  not 
include  material  cost.   For  each  of  these  models  the  coefficients  were 
estimated  for  several  different  sets  of  data  both  before  and  after 
WIPICS  and  for  both  the  engine  and  aircraft  programs. 

First,  a  period  of  400  days  of  data  well  before  the  installation 
of  WIPICS  was  chosen.   This  data  was  split  into  two  sections  composed 
of  the  first  300  and  last  100  days,  and  the  models  were  fit  to  both 
parts  to  investigate  the  stability  of  the  production  and  cost  function 
coefficients.   Since  the  400  day  period  was  well  before  the  implemen- 
tation of  WIPICS,  the  results  obtained  should  reveal  some  of  the  pro- 
blems to  look  for  in  the  estimation,  that  is,  the  expected  variability 
due  to  factors  other  than  WIPICS. 

Second,  data  periods  immediately  prior  to  WIPICS  and  after 
installation  and  breakin  of  WIPICS  were  used  to  again  estimate  the 
coefficients.   The  resulting  BW/AW  models  will  be  compared  in  section 
VI  to  see  if  WIPICS  had  an  impact  on  the  estimated  costs  of  the  aircraft 
and  engine  programs. 

The  results  of  these  estimations  and  the  stability  tests  will  be 
reported  in  this  section. 

B.   RESULTS  FOR  THE  BEFORE  WIPICS  PERIOD 

400  consecutive  days  of  before  WIPICS  data  covering  the  Julian  dates 
0191  -  1225  for  aircraft  and  0215  -  1250  for  engines  were  split  into 
the  first  300  and  last  100  observations  using  the  regression  procedures 
outlined  in  section  IV,  the  three  models  were  estimated  for  this  data. 
The  resulting  coefficients  are  shown  in  tables  V  through  X.   The  standard 
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error  of  each  estimated  coefficient  is  shown  in  parentheses  below  the 

2 
coefficient,   R   is  the  fraction  of  the  variance  of  the  dependent 

variable  explained  by  the  regression  equation,   SE   is  the  standard 

A 

error  of  the  dependent  variable,  and  p   is  an  estimate  of  the  serial 
correlation  of  the  random  error  terms  in  the  model. 

An  intuitive  examination  shows  that  the  coefficients  do  not  look 
very  similar  between  the  300  and  100  day  data  samples.   A  statistical 
measure  of  this  difference  can  be  obtained  using  a  statistical  technique 
known  as  the  Chow  test  [11,  pp  192-207].   Briefly,  the  Chow  test  is  as 
follows: 
Let 

0,  =  the  sum  of  squared  residuals  from  the  regression  on 

all  400  observations, 
Q  =  (the  sum  of  squared  residuals  from  the  regression  on 
the  first  300  observations) 
+ 

(the  sum  of  squared  residuals  from  the  regression  on 
the  last  100  observations) , 

Q3  =  Ql  "  Q2 

k  =  the  number  of  coefficients  estimated  in  each  regression. 
Then  under  the  hypothesis  that  the  coefficients  are  the  same  in  all  three 
regressions,  the  ratio 

F  = 


Q2/(400-2K) 
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is  distributed  as  an  F  statistic  with  k  and  400  -  2fc  degrees  of 
freedom.   If   F  >  F  (k,  400-2k) ,   then  reject  the  hypothesis  of  equality 
of  the  coefficients  at  the   e   confidence  level. 

The  Chow  tests  were  performed  on  the  Cobb-Douglas  production  and 
cost  functions  for  both  aircraft  and  engine  programs.   Details  of  the 
tests  are  given  in  Table  XI.   For  all  cases,  the  hypothesis  that  the 
coefficients  were  essentially  the  same  for  all  3  regressions  was  soundly 
rejected.   This  indicates  that  during  the  period  considered,  the  co- 
efficients of  the  regression  models  were  not  completely  stable.   Since 
the  data  on  which  these  regressions  are  performed  is  aggregated  time 
series  data,  it  is  wise  to  test  for  autocorrelation  of  the  residuals. 
This  was  done  for  the  Cobb-Douglas  cost  functions  (since  they  are  the 
models  we  will  use  the  most),  and  in  both  the  aircraft  and  engine  pro- 
grams  the  estimated  serial  correlation  coefficients   (p)  were  extremely 
high.   The  standard  first  order  autocorrelation  correction  [11,  p  260] 
was  made  for  this  data  and  the  regressions  were  repeated  on  the  corrected 
data  with  the  results  shown  in  Tables  XII  and  XIII. 

Chow  tests  run  on  these  corrected  models  still  rejected  the  hypo- 
thesis that  the  three  equations  were  statistically  identical  at  the  95% 
level,  but  the  computed   F  values  were  not  nearly  so  large  as  for  the 
uncorrected  models. 
C.   USE  OF  COST  FUNCTIONS  AS  PREDICTORS 

A  second  approach  to  validation  of  these  models  involves  assessing 
their  value  as  predictors.   This  approach  has  been  followed  for  the 
Cobb-Douglas  and  C.E.S.  cost  function  models.   For  each  of  the  cost 
function  models,  the  data  was  again  divided  into  the  first  300  and  the 


CHOW  TESTS 

Reject  =  if  F  >  F  (4,392)=  2.39 

(5%) 
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aircraft 

engine 

Cobb-Douglas 

production  function 

«i 

.037 

.065 

"2 

.032  +  .001 

.039  +  .007 

Cobb-Douglas 
cost   function 


11.9 


.031 


024  +  .001 


23.5 


40.5 


.143 


.067  +  .019 


65.0 


TABLE  XI 
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last  100  observations.   Cost  functions  were  estimated  by  regression 

using  the  first  300  days  data,  and  the  resulting  cost  equations  were 

used  to  predict  the  cost  for  each  of  the  last  100  days  of  data.   As  a 

criterion  for  comparison,  the  mean  squared  error  (MSE)  was  computed 

both  for  the  first  300  days  (upon  which  the  cost  equation  is  based)  and 

for  the  last  100  days  predicted  values. 

If   e.   is  the  error  of  the  predicted  cost  for  the  i    observation 
l 

a  confidence  interval  can  be  empirically  constructed  by  using  the  ratio 
of  two  standard  errors  of  the  function  over  the  actual  mean  costs  for 
the  period.   The  result  is  a  percent  error  in  cost, 


VXei/n 


%  error  -  — — x  100. 

y  c. 

Z^i      i  n 

The  %  error  approximates  +  2  standard  deviations  which  accounts  for 
about  90%  of  the  mass  in  most  probability  distributions.   Thus,  there  is 
approximately  a  90%  confidence  that  the  predicted  cost  is  within  the 
percent  error  computed  of  the  actual  cost. 

This  procedure  was  performed  on  the  cost  functions  estimated  from 
the  300  day  period,  and  then  on  the  100  day  period  with  costs  being 
predicted  by  the  300  day  models.   The  Cobb-Douglas  and  C.E.S.  cost 
functions  were  also  compared  to  the  "naive"  models 

C  =  3N 

C  =  a  +  3N 

which  predict  cost  based  only  on  norm,  to  determine  the  improvement  from 
using  the  cost  function  models.   The  results  for  the  Cobb-Douglas  cost 
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function  are  given  in  Table  XIV.   The  comparison  indicates  that  the 
Cobb-Douglas  Cost  function  does  a  substantially  better  job  of  both 
explaining  and  predicting  total  cost  than  the  naive  models  for  the 
aircraft  program.   For  the  engine  program  the  differences  among  the 
three  models  are  not  so  significant,  and  none  of  the  models  is  a  parti- 
cularly good  predictor  of  total  cost. 

For  the  C.E.S.  cost  function  a  similar  comparison  was  made.   Direct 
comparison  between  Cobb-Douglas  and  C.E.S.  is  not  easy  since  the  C.E.S. 
total  cost  does  not  include  material  cost  while  the  Cobb-Douglas  cost 
does.   The  results  for  the  C.E.S.  cost  function  are  given  in  Table  XV. 

Again,  the  C.E.S.  cost  function  is  superior  to  the  naive  models 
for  the  aircraft  program,  but  in  the  engine  program  this  is  not  the 
case. 

In  summary,  then,  both  the  Cobb-Douglas  and  the  C.E.S.  cost 
functions  do  a  good  job  of  explaining  and  predicting  total  cost  for 
the  aircraft  program.   Neither  Cobb-Douglas  nor  C.E.S.  offers  advantage 
over  naive  models  for  the  engine  program. 

A  similar  procedure  to  that  of  the  above  discussion  uses  statistical 
methods  outlined  in  Theil  [12,  pp  134-135]  to  construct  prediction 
intervals.    Let  C'  represent  the  estimated  cost  and  accept  the  assump- 
tions of  the  standard  linear  model.   Then  the  following  probability 
statement  holds: 


where 


P(£n  C'   -  H  <  in  C±  <    in  C±  +   H)  =  1-a 
H  =  ta/2(SE)  J   1  +  V»  (X'X)_1V0 
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aircraft 

engines 

Cobb-Douglas 

cost  function 

first  300  days 

+  4.0% 

+  7.0% 

last  100  days 

+  5.0% 

+  15.7% 

C  =  3  •  N 

first  300  days 
last  100  days 

C  =  ot  +  gN 
first  300  days 
last  100  days 


+  11.2% 
+  9.0% 


+  11.1% 
+  8.4% 


+  8.0% 
+  17.9% 


+  7.3% 
+  18.0% 


2\/MSE  expressed  as  percent  of  actual  cost. 


TABLE  XIV 
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Aircraft 


C.E.S.  cost  function 

first  300  days 
last  100  days 


C  =  pw 
first  300  days 
last  100  days 


C  =  a  +  3N 
first  300  days 
last  100  days 


+  2.3% 
+  3.1% 


+  4.8% 
+  7.2% 


+  4.8% 
+  6.6% 


Engines 


+   11.1% 
+  14.6% 


+  10.0% 
+  13.7% 


+   9.4% 
+   14.7% 


2  \/mSE  expressed  as  a  percent  of  actual  cost. 
(Note  total  cost  for  this  table  does  not  include  material  cost.) 


TABLE  XV 
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X  =  matrix  of  independent  variables  used  to 
construct  the  model 


Vn=  vector  of  independent  variables  used  to 


predict  £nC 


ta/2 


=  t-statistic 


This  interval  is  in  logarithms.   The  transformed  interval  in  costs 
becomes  non-symetric  and  non  linear  in  the  independent  variable  of  V_. 
Thus,  it  is  more  accurate  to  compare  the  extreme  widths  of  the  intervals 
than  the  mean  values  compared  in  subsection  1. 

Prediction  intervals  were  computed  for  the  100  day  period  using  the 
300  day  models.   Of  the  100  prediction  intervals  resulting  the  minimum 
and  maximum  interval  were  transformed  to  dollars  and  then  divided  by 
the  mean  cost  for  the  100  day  period  to  obtain  a  maximum  and  minimum  width 
as  a  percentage  of  the  average  cost  (parenthetical  entries  of  Table  XVI). 
Finally,  the  percentage  of  actual  costs  that  fell  outside  their  correspond- 
ing prediction  interval  is  reported  in  the  last  column  of  Table  XVI. 


Airframes 


Engines 


Model 

Max. 
Interval 

Min.       %  Out- 
Interval   side  Interval 

CD 
CES 

6729 
(8.5%) 

5365 
(6.8%) 

4% 

4480 
(4.0%) 

3347 
(3.0%) 

32% 

CD 
CES 

5375 
(12.5%) 

3783 
(8.8%) 

62% 

6943 
(20.5%) 

4836 
(14.3%) 

29% 

TABLE  XVI   Extreme  Widths  of  90%  Prediction 
Intervals  (in  dollars) 
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The  uneven  prediction  results  between  the  cost  functions  may  be 
indicative  of  the  Chow  Test's  conclusion  that  the  two  periods  of  data, 
300  vs.  the  following  100  days,  are  in  fact  different  and  a  model  of  one 
period  is  a  poor  prediction  of  costs  in  another  period.   This  possibility 
is  important  because  the  periods  are  not  separated  by  a  recognizable 
technological  change.   It  reemphasizes  the  need  to  include  the  accuracy 
of  the  models  in  the  final  comparison  of  estimated  costs. 

D.   ESTIMATED  MODELS  FOR  THE  BEFORE  WIPICS  -  AFTER  WIPICS  COMPARISON 

For  the  before  WIPICS  -  after  WIPICS  comparison  the  decision  was 
made  to  use  only  the  Cobb-Douglas  cost  function  corrected  for  auto- 
correlation. The  primary  reason  for  this,  is  that  autocorrelation  is 
clearly  a  problem  for  data  of  this  type.  Straightforward  corrections 
for  autocorrelation  exist  for  linear  regression  models  (such  as  the 
Cobb-Douglas  cost  function)  but  the  effect  of  autocorrelation  on  non- 
linear models  (such  as  the  C.E.S.  cost  function)  is  not  known. 

For  these  regressions  the  time  periods  were  chosen  as  close  as 
possible  to  the  technological  change  so  the  before  and  after  situations 
would  be  as  similar  as  possible  except  for  the  WIPICS  system.   The  obser- 
vations used  are: 


Aircraft 


Before  WIPICS 


After  WIPICS 


Number  of  Observations 


400  observations 


90  observations 


Julian  dates 


0316  to  1350 


2090  to  2179 


Engines 
Before  WIPICS 
After  WIPICS 


400  observations 


160  observations 


0247  to  1281 


2158  to  2317 
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different  periods  are  used  for  aircraft  and  engines  since  the  average 
length  of  stay  at  the  NARF  is  different  for  jobs  in  the  two  programs. 
The  resulting  Cobb-Douglas  cost  functions  (corrected  for  autocorrelation) 
are  presented  in  Tables  XVII  and  XVIII.   Comparison  of  these  cost  func- 
tions with  each  other  and  with  other  models  will  be  the  subject  of 
section  VI. 
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VI   COST  COMPARISONS  BEFORE  AND  AFTER  WIPICS 

A.  PROCEDURE 

The  cost  function  models  described  in  the  previous  sections  of  this 
report  were  used  to  compare  costs  on  typical  workloads  for  the  before 
and  after  WIPICS  situations.   The  results  of  these  cost  comparisons  are 
reported  in  this  section.   For  each  cost  equation  the  parameters  were 
estimated  twice,  once  using  before  WIPICS  data  and  once  using  after 
WIPICS  data.   The  resulting  two  cost  functions  will  be  called  the 
Before  WIPICS  cost  function  and  the  After  WIPICS  cost  function. 

Applying  the  Before  WIPICS  cost  function  to  a  particular  workload 
gives  an  estimate  of  what  the  cost  of  performing  that  work  would  have 
been  if  it  had  been  done  before  WIPICS  was  installed.   Similarly,  using 
the  After  WIPICS  cost  function  on  the  same  workload  gives  an  estimate 
of  what  the  cost  would  have  been  if  that  work  had  been  done  after 
WIPICS  was  installed.   The  difference  of  these  two  cost  estimates  yields 
an  indication  of  whether  WIPICS  was  cost-saving  for  that  particular  work- 
load.  Any  indicated  cost  savings  can  then  be  compared  with  WIPICS  costs 
to  see  if  WIPICS  is  cost-effective.   Most  of  the  results  reported  in  this 
section  will  deal  with  the  aircraft  program  since  even  though  WIPICS 
has  been  in  operation  for  some  time,  it  has  not  been  directly  applied 
to  the  engine  program.   For  the  Cobb-Douglas  cost  functions,  the  engine 
program  results  will  also  be  given  as  an  indication  of  possible  changes 
not  due  to  WIPICS. 

B.  CONTINUOUS  COST  FUNCTIONS  FOR  A  MAJOR  PROGRAM  (COBB-DOUGLAS) 

The  Before  and  After  WIPICS  Cobb-Douglas  cost  functions  for  both 
aircraft  and  engines,  corrected  for  autocorrelation,  were  given  in 
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section  V  -  D.   The  actual  workloads  for  the  NARF  during  the  Before 
and  the  After  WIPICS  periods  were  used  as  representative  workloads  for 
the  cost  comparison.   The  results  are  given  in  tables  XIX  and  XX. 
The  costs  indicated  in  the  tables  include  estimated  cost  of  labor,  material, 
overhead,  and  penalty  for  time  in  rework.   Since  the  Cobb-Douglas  cost 
functions  operate  on  prorated  aggregated  daily  data,  the  cost  estimates  re- 
present the  average  daily  total  cost  for  all  jobs  in  the  shop  over  the 
time  period  being  used. 

Positive  values  of  differences  indicate  a  cost  savings  while  negative 
differences  indicate  that  performing  the  same  work  After  WIPICS  would 
cost  more  than  if  it  had  been  done  Before  WIPICS. 

Detailed  discussion  of  these  results  will  be  postponed  to  section 
VI  -  E,  after  the  results  from  all  the  different  cost  functions  are  avail- 
able. 

C.   LINEAR  ECONOMIC  MODEL  COST  FUNCTION 

The  linear  economic  model  of  section  II  -  D  was  used  to  estimate 
Before  WIPICS  and  After  WIPICS  costs  for  the  aircraft  program.   837  air- 
craft jobs  were  separated  into  three  groups: 

1.  414  jobs  before  WIPICS 

2.  158  jobs  in  a  buffer  period 

3.  265  jobs  after  WIPICS 

The  buffer  period  consisted  largely  of  jobs  started  before  and  finished 
after  WIPICS  was  initiated.   That  period  included  all  jobs  with  induction 
dates  between  Julian  dates  1181  and  1334  inclusive.   These  dates  were 
arbitrarily  chosen  to  allow  for  the  long  time  period  required  for  aircraft 
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processing  plus  the  indeterminate  period  while  WIPICS  was  being  initially 
tested.   The  before  and  after  groups  were  the  same  as  those  used  to 
generate  the  regressions  used  in  the  job  cost  model. 

The  two  groups  were  compared  and  aircraft-work  categories  common  to 
both  periods  were  selected  for  the  cost  comparison.   The  jobs  in  these 
20  common  categories  were  then  selected,  representing  70.4%  (257/365)  and 
68.7%  (182/265)  of  the  original  before  and  after  WIPICS  groups,  respec- 
tively.  In  the  interest  of  descriptive  brevity,  let 
| B  [  be  the  set  of  observations  before  WIPICS 
JB  i  be  the  subset  of  observations  of  |  B  j  in  the  20  common 

categories 
|  a}  be  the  set  of  observations  after  WIPICS 
iA^l  be  the  subset  of  |a[  that  matches  the  categories 

of  {bm( 

Activity  vectors  and  objective  function  coefficients  were  then  estimated 
for  both  data  sets  Jb  }  and  JA^J  using  the  procedures  developed  by 
Myers  [2].   The  resulting  linear  economic  models  were  formulated  with 
three  different  objective  functions. 

1)  Min:   Total  Cost  =  [pTT2  +  CTT3]  Z 

T 

2)  Min:   Operations  Cost  =  P  I^L 

T 

3)  Min:   Penalty  Cost  =  C  I^L 

In  order  to  compare  the  models,  two  test  cases  were  run  against  both  the 
before  and  after  models.   The  test  cases  consisted  of  the  actual  jobs  in 
{B  (  and  j  A  I  and  the  two  price  vectors  associated  with  them.   Activity 
and  price  vectors  run  against  their  own  models  simply  provide  actual 
cost  determinations,  but  the  runs  against  the  opposite  model  represent 
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predicted  costs.   The  costs  were  calculated  for  each  of  the  objective 
function  versions  and  the  results  are  listed  in  Tables  XXI,  XXII,  XXIII. 
The  costs  in  these  tables  are  the  estimated  total  costs  of  doing  all  the 
jobs  in  the  workload  in  question.   Figures  are  listed  in  millions  of 
dollars.   The  upper  right  corner  prediction  of  Table  XXI,  for  example, 
is  read,  "If  the  jobs  represented  by  {  K.\     had  been  accomplished  during 
the  before  WIPICS  period,  it  is  predicted  that  the  total  cost  would  have 
been  $40,799  million." 

As  the  comparative  results  indicate,  the  after  WIPICS  period  costs 
were  estimated  to  be  higher  in  each  of  the  cost  variations  used.   Com- 
parison of  entries  in  the  T„  matrices  for  before  and  after  situations 
emphasized  the  increases  in  resources  used  per  hour  of  NORM.   In  every 
process  at  least  one  coefficient  was  higher  for  the  after  WIPICS  period, 
with  many  cases  showing  two  and  three  increased  coefficients.   Any  de- 
creases were  relatively  smaller  in  magnitude  than  the  increases.   This 
model  contains  no  inherent  inflation  compensation  for  the  materials  and 
overhead  expenditures  such  as  the  DMHR  price  has. 

Comparing  the  activity  vectors  before  and  after  WIPICS  highlighted 
another  potential  problem.   The  two  vectors  presented  radically  different 
levels  of  activity  in  many  of  the  processes.   Such  differences  could 
greatly  magnify  any  deviation  of  the  linear  relation  from  the  true 
relationship,  which  is  more  probably  non-linear.   The  coefficients  used 
in  these  linear  models  and  the  details  of  the  {  BM  [  and   {  A^  }  work- 
loads can  be  found  in  McGarrahan  [6], 
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D.   CONTINUOUS  COST  FUNCTION  FOR  A  JOB 

The  continuous  cost  function  for  a  job  was  developed  in  section 
IV-D.   Two  versions  of  this  cost  function  were  estimated  and  compared 
using  the  same  before  and  after  WTPICS  workloads  |b[  and  Ja|  as 
in  the  linear  model. 

The  two  versions  of  the  cost  function  used  are: 

Bl   62  33   64 

1)  C  ■  A.  P.   P.   HIS   H 

1  L    D 

h     62  B3   64 

2)  C  =  A.  PT    T   NIS   N 

x  L 


where  NIS   is  the  average  number  of  jobs  in  the  shop  over  the  time  period 
when  this  job  was  in  the  shop.   T   is  the  length  of  time  this  job  stayed 
in  the  shop,   P   is  production  date,  and  all  other  variables  are  as  pre- 
viously defined. 

A  total  of  ten  relationships  were  estimated  from  the  data  in  {b| 
and  j  A  \     .   These  represented  before  and  after  WIPICS  characterizations 
of  the  following: 

1.   Version  1  2.   Version  2 

a.  Total  Cost  a.   Total  Cost 

b.  Operations  Cost  b.   Operations  Cost 

c.  Penalty  Cost 

The  estimation  was  done  only  for  the  aircraft  program.   Details  of  the 
estimated  coefficients  and  statistical  properties  of  these  models  can 
be  found  in  McGarrahan  [6].   The  resulting  cost  functions  were  then 
applied  to  the  jobs  in  data  sets  i  B  J  and  J  A^  J.  (recall  these  contain 


job  categories  which  are  present  both  before  and  after  WIPICS)  to  obtain 
cost  comparisons.   The  resulting  comparisons  are  given  in  Tables  XXIV 
through  XXVIII.   Figures  in  the  tables  are  estimated  total  cost  to  perform 
all  the  jobs  in  the  given  workload.   Costs  are  in  millions  of  dollars. 
Positive  differences  indicate  predicted  cost  savings  after  WIPICS,  nega- 
tive differences  indicate  predicted  cost  increases  after  WIPICS. 

The  results  do  not  seem  to  indicate  consistent  superiority  of  one 
cost  function  over  the  other.   Further  discussion  of  these  results  will 
be  included  in  the  next  section. 
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E.   SUMMARY  OF  COST  COMPARISONS 

The  cost  difference  for  the  various  cost  functions  are  brought 
together  in  Table  XXIX.   The  differences  reported  are  all  predicted 
cc    ^-.m  before  WIPICS  cost  function  minus  predicted  cost  from  after 
WIPICS  cost  function.   For  the  Cobb-Douglas  models  the  differences  are 
expressed  as  dollars  per  day  while  for  the  other  linear  and  job  models 
the  differences  are  in  millions  of  dollars  for  completing  the  workload, 
Positive  values  indicate  cost  savings  negative  values  indicate  cost 
increases. 
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VII.   SUMMARY  AND  CONCLUSIONS 

This  report  has  explored  several  alternative  approaches  to  the  problem 
of  auditing  technological  change.   Three  distinct  types  of  models  have  been 
documented  and  several  versions  of  some  types  have  been  examined  in  detail 
and  applied  to  the  problem. 

The  linear  economic  model  is  the  most  detailed  of  the  model  types.   It 
permits  different  relations  between  costs,  inputs,  and  output  for  each  dif- 
ferent job  description.   This  model  type  has  several  drawbacks  however. 
Estimation  procedures  for  the  model  are  not  well  developed  and  the  statis- 
tical properties  of  the  estimators  are  unknown.   The  model  requires  large 
quantities  of  data.   It  also  requires  the  restrictions  of  constant  returns 
to  scale  and  independence  of  production  processes;  two  assumptions  which 
are  unlikely  to  hold  in  any  given  application. 

The  second  model  type  is  based  on  a  continuous  production  function  for 
a  program.   The  model  uses  data  that  is  aggregated  over  jobs  and  is  there- 
fore not  affected  by  the  fact  that  individual  jobs  may  not  be  independent 
of  one  another.   One  of  the  models,  based  on  the  Cobb-Douglas  production 
function,  is  able  to  be  straightforwardly  estimated  by  statistics  with  well 
known  properties  in  the  absence  of  perverse  circumstances.   This  model  also 
has  the  advantage  of  not  imposing  a  requirement  of  constant  returns  to  scale. 
On  the  other  hand  the  model  is  not  as  sensitive  to  changes  in  workload  as  the 
others.   It  also  relies  heavily  on  production  load  norm  as  a  measure  of  out- 
put, at  best  a  relatively  imprecise  measure.   The  advantages  of  using  aggre- 
gated data  in  the  model  are  balanced  by  the  fact  that  the  aggregated  data 
set  tends  to  be  autocorrelated.   Thus  a  special,  though  not  unusual,  correction 
must  be  applied  in  using  the  model. 
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The  models  based  on  a  continuous  production  function  for  a  job  combine 
some  of  the  characteristics  of  the  previous  models.   These  models  are  able 
to  account  for  changes  in  the  mix  of  jobs  without  imposing  the  restriction 
of  constant  return  to  scale.   They  also  permit  estimation  by  statistics 
with  well  known  properties.   The  models  do  not  use  aggregated  data  and  there- 
fore present  no  problem  of  autocorrelation.   However,  these  models  do  suffer 
from  the  fact  that  each  job  is  assumed  to  be  independent  of  any  other  and  as 
a  result  errors  are  introduced  when  this  assumption  is  violated. 

Several  versions  of  each  of  the  model  types  were  applied  to  data  supplied 
by  NARFNI  for  periods  of  time  both  before  and  after  the  introduction  of  the 
WIPICS  system.   The  results  of  the  application  are  reported  in  Chapter  VI  and 
briefly  summarized  in  Table  XXIX. 

The  results  in  Table  XXIX  apply  only  to  the  aircraft  program  because  the 
WIPICS  system,  while  operating,  had  not  been  applied  to  the  engine  program 
at  the  time  data  was  collected,  March  1973. 

Even  a  brief  glance  at  Table  XXIX  indicates  that  the  results  are  not 
conclusive.   Only  the  two  versions  of  the  last  model  type  indicate  cost  savings 
for  the  WIPICS  system  in  excess  of  the  direct  costs  of  the  system  and  even  here 
there  is  significant  disagreement  when  applied  to  the  before  WIPICS  workload. 
The  other  model  types  indicate  that  the  costs  of  the  system  exceed  the  cost 
savings  generated  by  it. 

Several  general  conclusions  may  be  drawn  from  the  study.   Obviously,  the 
evidence  that  now  exists  is  insufficient  to  conclude  that  the  WIPICS  system  is 
cost  effective  at  this  time.   The  results  of  the  various  regression  analyses 
are  too  contradictory  to  warrant  such  a  conclusion.   This  result  compares 
quite  favorably  with  other  evidence  which  indicates  that  the  system  at  this 
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time  had  not  been  fully  implemented  on  the  airframe  program.   In  addition 
it  must  be  emphasized  that  a  system  like  WIPICS,  if  it  is  to  be  useful  and 
to  generate  cost  savings,  must  be  faithfully  used  by  a  large  number  of  people 
in  many  different  shops  throughout  the  NARF.   Discussions  with  the  management 
of  the  NARF  seem  to  indicate  that  during  the  time  period  involved  many  people 
were  not  sufficiently  knowledgeable  to  use  the  system  effectively.   As  a 
result,  the  accuracy  of  the  data  in  the  system  was  suspect  and  the  usefulness 
of  the  system  questionable.   These  conclusions  are  drawn  primarily  from  infor- 
mation provided  by  the  Naval  Area  Audit  Service. 

The  lack  of  agreement  among  the  various  models  used  illustrates  the  effect 
of  different  underlying  assumptions  and  differing  model  characteristics.   There 
is  no  objective  way  to  choose  among  these  alternative  models.   However,  the 
continuous  models  do  present  statistics  which  enable  one  to  construct  confidence 
intervals  and  test  hypotheses  about  the  importance  of  variables.   Thus  there 
is  some  tendency  to  find  the  results  of  these  models  more  believable  than  the 
results  of  the  linear  economic  model.   Even  among  the  continuous  models  there 
is  disagreement  however.   The  problems  here  stem  mainly  from  the  fact  that  the 
data  are  inherently  variable.   The  continuous  cost  functions  which  were  esti- 
mated while  explaining  large  proportions  of  the  variations  in  costs  are  still 
accurate  to  within  only  5  to  10  percent  of  total  costs.   Differences  among  the 
alternative  regression  analyses  are  well  within  these  bounds.   This  emphasizes 
the  fact  that  no  definitive  conclusions  concerning  the  cost  effectiveness  of 
the  WIPICS  system  on  the  airframe  program  may  be  drawn  at  this  time. 

The  possible  sources  of  this  unexplained  variation  in  total  costs  are 
numerous.   Some  of  them  are  discussed  briefly  below. 

Measurement  of  the  variables.   As  discussed  in  Chapter  Three,  the  various 
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regression  analyses  use  several  proxy  variables  to  measure  prices,  inputs 
and  outputs  of  the  production  activity  at  the  NARF.   This  use  of  proxy 
variables  to  measure  desired  variables  will  of  course  inject  unknown  errors 
into  the  analyses.   In  particular  the  treatment  of  material  cost  as  a  quan- 
tity, thus  ignoring  price  increases  for  material  may  have  been  an  important 
source  of  such  errors.   This  particular  source  of  error  was  to  some  extent 
adjusted  for  by  including  the  production  date  as  a  variable  in  version  one 
of  the  job  cost  model. 

Collection  of  the  data.   Two  particular  types  of  data  collection  pro- 
blems are  possible  sources  of  unexplained  variation.   One  problem  arises 
from  the  fact  that  in  some  instances  production  load  norm  is  renegotiated 
after  a  particular  repair  job  is  begun.   While  the  renegotiated  norm  may 
be  more  representative  of  the  actual  work  to  be  done  it  seems  likely  that 
the  series  of  norms  some  renegotiated  and  some  not,  bear  a  random  relation- 
ship to  work  accomplished.   No  attempt  was  made  to  isolate  those  jobs  with 
renegotiated  production  load  norms  for  special  analysis* this  may  be  a 
source  of  unexplained  variations. 

Another  data  collection  problem  concerns  the  allocation  of  material 
cost  to  jobs  and  impacts  on  the  assumption  that  different  jobs  in  the  NARF 
are  independent  of  one  another.   In  some  cases  "job  a"  needs  a  part  which 
is  not  available.   To  expedite  the  aircraft  a  used  part  from  "job  b"  is 
placed  on  "job  a"  and  a  new  part  ordered  for  "job  b".   By  this  process 
"job  a"  is  completed  earlier  than  it  would  have  been  and  "job  b",  which 
has  many  additional  operations  to  be  performed  on  it,  is  completed  no  later 
than  it  would  have  been.   The  problem,  from  the  point  of  view  of  data  collec- 
tion, is  which  job  gets  charged  for  the  new  part.   Unfortunately,  the  answer 
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to  this  question  is  not  uniform.   In  some  cases  the  job  which  needed  the 
part  "job  a"  is  charged  for  the  new  part  in  other  cases  the  job  which 
actually  received  the  new  part  "job  b"  is  charged.   This  problem  impacts 
on  man  hour  data  as  well.   Here  the  question  is  which  job  should  be  charged 
the  labor  cost  of  removing  the  used  part  from  "job  b"  and  of  replacing  it 
with  the  new  part. 

The  existence  of  backrobbing  tends  to  argue  that  jobs  in  the  shop  are 
not  independent  of  one  another.   Although  backrobbing  may  be  too  infrequent 
to  incorporate  into  the  model  explicitly,  it  remains  a  source  of  unexplained 
variation. 

Assumptions  and  form  of  the  relations.  Any  attempt  to  describe  the  be- 
havior of  a  complex  organization  with  a  relatively  simple  relation  involving 
only  a  few  variables  is  bound  to  be  subject  to  some  error.   The  relations 
estimated  in  this  report  are,  of  course,  not  exceptions  to  this  general  rule 
and  thus  some  unexplained  variation  in  the  data  is  to  be  expected.   While 
it  is  not  desirable  to  complicate  the  relations  to  the  point  of  explaining  every 
detail  of  the  operations  there  are  several  influences  on  production  costs  which 
may  require  explicit  treatment.   The  effects  of  changes  in  retirement  policy, 
the  impositions  of  personnel  ceilings,  and  various  admonitions  and  constraints 
placed  on  the  NARF  have  been  ignored  in  the  cost  functions  which  have  been 
estimated.   These  omissions  as  well  as  others  are  also  sources  of  unexplained 
variation  in  costs  in  the  analyses. 

Despite  the  fact  that  there  are  many  factors  left  out  of  the  regression 
equations,  the  fact  that  variables  were  measured  subject  to  error  and  the  fact 
that  proxy  variables  were  used;  each  of  the  continuous  cost  functions  derived 
above  explain  over  ninety  percent  of  variation  in  total  costs  of  production 
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in  the  aircraft  and  engine  programs  at  the  NARF.   Also,  while  there  are  dif- 
ferences in  the  prediction  of  total  costs  of  production  among  the  various  con- 
tinuous models  an  analysis  of  each  requires  the  conclusion  that  the  net  cost 
savings  of  the  WTPICS  system  at  this  point  in  time  are  not  significantly 
different  from  the  usual  random  variations  in  costs  experienced  at  the  NARF. 
This  conclusion  agrees  with  other  observations  on  the  system  provided  by 
independent  sources. 
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